# Clinical decision supportdecision supportTechnologies and processes that turn raw data into actionable insights via reporting, dashboards and analysis, so teams can decide based on facts rather than intuition.Voir la définition complète → and diagnostic imaging that clinicians trust
A sepsis alert fires at 3 a.m. The nurse glances at the screen, taps "acknowledge," and keeps walking. She has seen this alert fire dozens of times this week, and most of the time the patient was fine. This is the central problem of clinical AI: a model can be statistically impressive and clinically useless if nobody at the bedside acts on it.
This lesson dissects two real-world tool types (a sepsis-prediction alert and a radiology triage model) to show why the metrics that vendors advertise are rarely the metrics that decide adoption.
Start with a definition. Accuracy is the share of all predictions a model gets right. In healthcare, it is almost useless on its own, because most patients do not have the condition you are screening for.
Imagine 1,000 admitted patients, and 20 develop sepsis (a life-threatening response to infection). A model that simply predicts "no sepsis" for everyone is 98% accurate. It also misses every single septic patient.
So clinicians ignore accuracy and ask about two other numbers:
These two trade off against each other. Push a sepsis model to catch nearly every case (high sensitivity), and you generate a flood of false alerts (low specificity). That flood creates the 3 a.m. problem.
Sepsis is a good test case because early treatment saves lives, but the early signs (rising heart rate, mild fever, subtle lab shifts) overlap with dozens of harmless conditions.
Alert fatigue is what happens when clinicians receive so many notifications that they start dismissing them reflexively, including the real ones. It is one of the most documented failure modes in hospital IT.
A widely reported example: an early commercial sepsis prediction model deployed across many U.S. hospitals was later found in an external study to perform far worse in practice than advertised, missing a large share of sepsis cases while still generating many alerts. You can read a summary of that scrutiny in this JAMA Internal Medicine research letter. The lesson was not "AI does not work." It was that a model validated in one setting can degrade badly in another, and that alert burden was never properly measured.
Three things, in order:
1. Positive predictive value (PPV). Of the times the model cries "sepsis," how often is it right? This is what the nurse experiences directly. A model with 90% sensitivity but 5% PPV means 19 out of 20 alerts are noise. Clinicians learn that ratio within a week and start ignoring the tool.
2. Lead time that is actionable. An alert that fires two hours before clinical deterioration lets a team draw cultures, start fluids, and begin antibiotics. An alert that fires when the patient is already crashing adds nothing.
3. Workflow integration. Does the alert land inside the electronic health record (EHR) where the clinician already works, with a one-click order set attached? Or does it require logging into a separate dashboard nobody opens? The best model in the world fails if it lives in the wrong place.
Below is the kind of threshold analysis a hospital data team runs before deployment. It is not the model itself, just the business logic of choosing an alert cutoff.
# Choosing a risk-score threshold for a sepsis alert.
# Goal: catch most sepsis cases without drowning nurses in false alarms.
for threshold in [0.3, 0.5, 0.7]:
flagged = risk_scores >= threshold
sensitivity = (flagged & has_sepsis).sum() / has_sepsis.sum()
ppv = (flagged & has_sepsis).sum() / flagged.sum()
alerts_per_day = flagged.sum() / days_in_data
print(threshold, round(sensitivity, 2), round(ppv, 2), round(alerts_per_day, 1))
# A low threshold catches more sepsis but floods the unit.
# A high threshold is quiet but misses cases.
# The chosen cutoff is a clinical decision, not a purely technical one.The point: the threshold is negotiated with the clinical staff who will live with the consequences, not set by the vendor.
Now flip to imaging. Here the AI often does not diagnose at all. It triages: it reorders the radiologist's worklist so urgent scans are read first.
Consider a model that scans head CTs for signs of intracranial hemorrhage (bleeding in the brain). Several such tools have received U.S. Food and Drug Administration (FDA) clearance as computer-aided triage devices, meaning they are cleared to prioritize cases, not to make the final call.
Why does this framing matter? Because the model is not replacing the radiologist. It is answering one question: "which of these 40 pending scans should a human look at in the next five minutes?"
For triage, high sensitivity matters enormously. Missing a brain bleed is catastrophic. A few false positives are tolerable, because a radiologist reviews every flagged scan anyway and can quickly dismiss the ones that are clear. The human backstop changes the acceptable error balance.
Radiology triage tools have generally seen smoother adoption than predictive alerts, for structural reasons:
Contrast that with the sepsis alert, which interrupts a busy nurse, cannot be visually verified, and adds a decision to an already overloaded shift.
Vérification des acquis
1. Why is accuracy considered a misleading metric for evaluating a sepsis-prediction model?
2. A clinician wants to ensure a sepsis model misses as few true cases as possible. Which metric should they prioritize, and what is the likely trade-off?
3. What is the core lesson illustrated by the nurse who taps 'acknowledge' on the 3 a.m. sepsis alert and keeps walking?
4. Select ALL correct answers about the relationship between sensitivity and specificity in clinical models.
Sélectionnez toutes les réponses correctes.
5. Select ALL correct answers describing why sepsis is a challenging condition for AI prediction models.
Sélectionnez toutes les réponses correctes.
Pull the two cases together. When evaluating any clinical AI tool, the questions that predict real-world success look nothing like a leaderboard score.
A model trained on one health system's population may fail on another's because of different patient mix, different lab equipment, or different documentation habits. This is called distribution shift. Ask for evidence of local validation, ideally a silent trial (the model runs in the background and its predictions are compared to reality before anyone acts on them).
Every alert is a claim on a human's attention. MapMapUsing software to automate repetitive marketing tasks and campaigns, enabling personalisation at scale across channels like email, web, and social.Voir la définition complète → the workflow. If the tool adds clicks without removing any, adoption will stall regardless of performance.
Regulators and clinicians both favor tools where a person makes the final decision. The FDA's framework for Software as a Medical Device (SaMD) distinguishes tools by how much they drive clinical action. Higher autonomy means higher scrutiny.
Models drift as patient populations, treatments, and coding practices change. A tool without ongoing monitoring is a liability waiting to surface. Ask who watches the model's PPV and sensitivity six months in, and what triggers a retrain or shutdown.
For a hospital executive, the takeaway is financial as much as clinical. A tool with excellent published accuracy but poor workflow fit generates:
The value is not in the algorithm. It is in the fraction of alerts that change a decision and improve an outcome. That is the number to negotiate contracts around, and it is rarely on the vendor's first slide.