Why Accurate AI Fails in Hospitals, and How Mayo Clinic Made It Work

That question sat at the centre of my conversation on The Chief Healthcare Officer podcast with Dr Suchi Saria, founder and CEO of Bayesian Health and professor at Johns Hopkins. Bayesian Health and Mayo Clinic had just announced an AI tool that identifies hospitalised patients who could benefit from palliative care earlier, before a crisis forces the conversation.

Why the company exists

Suchi spent years at Johns Hopkins building AI for the early detection of complications such as sepsis. Then she lost her nephew to sepsis. The gap between what the science could do and what reached the bedside in time became the reason for Bayesian Health. Its platform now runs inside Epic, Cerner and Allscripts across health systems caring for more than 25 million patients, learning what "normal" looks like for each individual patient rather than firing static rule-based alerts.

The Mayo Clinic results

Mayo's palliative care team had found through chart reviews that roughly one in two returning patients with advanced illness would have benefited from earlier palliative care. What they lacked was a way to find those patients across a whole system in real time. Mayo brought clinical expertise and validation; Bayesian brought the platform inside the existing workflow. Once live:

• 44% more timely palliative care referrals
• 25% fewer 60-day readmissions and 28% fewer 90-day readmissions
• Nearly five days shorter length of stay

From 2% to 85% adoption

The number that stayed with me came from an earlier sepsis rollout in 2018. First attempt: two physicians used it. Second attempt, over two years: 1 to 2% adoption. The hospital's chief digital officer called it an abject failure and assumed his clinicians were simply more sceptical than most.

They were not. What changed was the approach, not the algorithm. Clinicians trust a system that listens and adapts instead of repeating an ignored alert, that shows its reasoning instead of acting as a black box, and that supports their workflow before, during and after a decision. Applying those principles, the same hospital reached 85% adoption in six months.

Quality is the financial strategy

In a margin-squeezed system, quality is not a cost competing with financial sustainability. It is the financial strategy. Every avoided readmission, every day of stay saved and every complication caught early is better for the patient and for the system's survival. For one 2,000-bed system, addressing five or six such conditions was worth roughly $100 million in margin.

What to carry into your AI strategy

• Start from your own data to find the real opportunity, not from the vendor with the most polished demo.
• Design deliberately for clinician trust; it does not follow automatically from accuracy.
• Build a roadmap that lets teams win small before winning big. Adoption compounds.
• Insist on genuine clinical partnership and validation from any AI vendor, not just a technology sale.

Questions for boards and leadership teams

1. For each AI tool we have bought, what percentage of intended users actually use it?
2. Do we measure AI by model accuracy, or by changed clinical decisions and outcomes?
3. Which five conditions, caught earlier, would most improve both outcomes and margin?
4. Who in our organisation is responsible for clinician trust in AI?

What changed was not the algorithm. It was the approach.

Listen to the full episode at chiefhealthcareofficer.com.

First published in The Chief Healthcare Officer newsletter on LinkedIn.

Related reading: AI in hospitals: policy before pilot · The Coming Wave