Mapping the regulatory landscape for hospital AI
A sepsis-prediction tool flags a patient in Bed 12 as high-risk. That single alert just touched four different regulatory regimes. The FDA cares whether the algorithm is a cleared medical device. HIPAA cares how the patient's data moved to train and run it. ONC transparency rules care whether the clinician can see how the score was built. And the Joint Commission cares whether your hospital governed the whole thing responsibly.
Nobody hands you a mapmapUsing software to automate repetitive marketing tasks and campaigns, enabling personalisation at scale across channels like email, web, and social.View full definition → showing which rule binds which part. This lesson draws it.
The four regimes at a glance
Think of four overlapping circles, each owned by a different body.
- FDA (Food and Drug Administration): regulates the software itself when it qualifies as a medical device.
- HHS Office for Civil Rights: enforces HIPAA (Health Insurance Portability and Accountability Act), the privacy and security rules for patient data.
- ONC (Office of the National Coordinator for Health IT): sets algorithm transparency requirements inside certified electronic health record (EHR) software.
- The Joint Commission: the main hospital accreditation body. Not a government agency, but losing accreditation can end Medicare payment, so hospitals treat it as binding.
They overlap. A single deployment can trigger all four. The trick is knowing which trigger fires when.
FDA: is your sepsis tool a regulated device?
The FDA's domain is SaMD (Software as a Medical Device): software that performs a medical function without being part of a hardware device.
The key question: does the software diagnose, treat, or drive a clinical decision? A sepsis model that outputs a risk score to guide treatment is very likely SaMD. A tool that only summarizes existing chart notes for a clinician to read may not be.
Most predictive clinical tools clear the FDA through the 510(k) pathway, which shows the device is "substantially equivalent" to one already on the market. Higher-risk tools may need the stricter De Novo or PMA (Premarket Approval) routes.
Two traps for hospitals:
- Home-grown models. If your data science team builds a sepsis model in-house and uses it only inside your own hospital, FDA has historically exercised enforcement discretion, meaning it often does not require clearance. Buy the same capability from a vendor and it usually needs clearance. Same math, different rule, because the trigger is commercial distribution.
- The model that keeps learning. Traditional clearance assumes a frozen algorithm. For models that update over time, the FDA introduced the Predetermined Change Control Plan (PCCP): you pre-declare what the model is allowed to change and how you will validate it, so retraining does not require a new submission each time.
You can search cleared AI devices in the FDA's public list of AI-enabled medical devices.
HIPAA: the data underneath
HIPAA does not care whether your model is clever. It cares about PHI (Protected Health Information): identifiable patient data.
Where HIPAA bites in an AI project:
- Training data. Using real patient records to train the sepsis model is a "use" of PHI. It must fall under treatment, payment, operations, or be de-identified (stripped of the 18 HIPAA identifiers such as name, dates, medical record number).
- Vendors. If a vendor touches PHI, you need a Business Associate Agreement (BAA): a contract making them liable for protecting the data. No BAA, no data sharing. Full stop.
- Cloud inference. Sending Bed 12's vitals to a cloud APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition → for scoring is a transmission of PHI. It needs encryption and a BAA with the cloud provider.
A common failure: a data scientist pipes live records into a general-purpose large language modellarge language modelA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.View full definition → with no BAA to "test an idea." That is a reportable breach waiting to happen.
ONC: transparency inside the EHR
ONC regulates certified health IT, which is the EHR software (Epic, Oracle Health, and others) that hospitals rely on for Medicare and Medicaid programs.
Under the HTI-1 rule (Health Data, Technology, and Interoperability, finalized 2024), certified EHRs that surface predictive decision support interventions (DSIs), meaning AI or algorithmic recommendations, must expose a set of "source attributes." In plain terms, the clinician (or the hospital) must be able to see a kind of nutrition label for the algorithm:
- What data it was trained on.
- What outcome it predicts.
- How its performance was validated.
- Known limitations and fairness considerations.
So if your sepsis score shows up inside Epic, HTI-1 governs whether users can pull up that label. This is transparency, not approval. ONC does not say the model is good. It says you must be able to see what it is.
Joint Commission: governing the deployment
The Joint Commission accredits hospitals and, as of 2025, published responsible use of AI guidance for healthcare organizations. Its focus is not the algorithm, it is your process around it:
- Do you have governance oversight of AI tools?
- Do you monitor the model after go-live for drift and bias?
- Do staff know when a recommendation came from AI?
- Is there a way to report and act on AI-related safety events?
This is where the other three regimes get operationalized. The FDA clears the tool, HIPAA protects the data, ONC exposes the label, and the Joint Commission checks that your hospital actually runs the guardrailsguardrailsRules and controls that keep an AI system inside safe, legal and on-brand boundaries, blocking outputs and actions that cross the line.View full definition →.
Putting it on one tool
Here is the sepsis model, mapped:
| Regime | What it governs | Trigger |
|---|---|---|
| FDA | The algorithm as a device | Sold by a vendor, drives clinical decisions |
| HIPAA | Patient data used and moved | Any use of identifiable PHI |
| ONC HTI-1 | Transparency label in the EHR | Surfaced through certified health IT |
| Joint Commission | Hospital governance and monitoring | Any AI used in accredited operations |
Notice: an in-house model skipping the EHR might dodge FDA clearance and ONC labeling, but it still faces HIPAA and the Joint Commission. Regimes overlap, they do not substitute.
A minimal pre-deployment checklist as pseudocode:
for tool in ai_deployments:
if tool.drives_clinical_decision and tool.is_vendor_supplied:
require FDA_clearance(pathway in [510k, DeNovo, PMA])
if tool.uses_PHI:
require de_identification OR (BAA_signed and encryption_enabled)
if tool.surfaced_in_certified_EHR:
require HTI1_source_attributes_available
require JointCommission.governance(monitoring, bias_check, staff_disclosure)Knowledge check
1. A hospital deploys a tool that only compiles and summarizes existing chart notes for a clinician to read, without producing risk scores or treatment guidance. Why is this tool less likely to be regulated as SaMD by the FDA?
2. The lesson notes that the Joint Commission is 'not a government agency, but hospitals treat it as binding.' What is the best explanation for this apparent contradiction?
3. A single sepsis-prediction alert can implicate four regulatory regimes at once. What is the core conceptual point the lesson makes about these regimes?
4. Select ALL correct answers. Which regulatory concerns would be triggered by a sepsis model that produces a risk score inside a certified EHR using patient data?
Select all the correct answers.
5. Select ALL correct answers. Which statements correctly describe the FDA's role and the 510(k) pathway as presented in the lesson?
Select all the correct answers.
Where teams actually get tripped up
"It's just decision support, so FDA doesn't apply." Sometimes true, but if the tool nudges treatment and comes from a vendor, assume SaMD until proven otherwise. The intended-use language in your marketing matters as much as the code.
"We de-identified, so HIPAA is done." De-identification must meet the HIPAA standard (Safe Harbor removal of 18 identifiers, or Expert Determination). Dropping only the patient name is not enough.
"The vendor is FDA-cleared, so we're covered." Clearance covers the device. It does not cover your monitoring, your BAA, or your EHR transparency. Those are yours.
"HTI-1 means the model is approved." No. ONC transparency is about visibility, not endorsement. A poorly performing model can still show its label.
Europe in one line
For readers deploying across the Atlantic: the EU AI Act (in force from 2024, with obligations phasing in through 2026 and 2027) classifies most clinical AI as high-risk, layering conformity assessment on top of existing MDR (Medical Device Regulation) device rules. The structure differs, but the instinct is the same: prove the tool is safe, keep the data lawful, and govern the deployment.
Key Takeaways
- One tool, four regimes. A sepsis model can trigger FDA (device), HIPAA (data), ONC (transparency), and Joint Commission (governance) at once. Map each before go-live.
- The FDA trigger is intended use plus distribution. Vendor-sold tools that drive clinical decisions usually need clearance; some in-house tools get enforcement discretion, but that is not permission to skip governance.
- HIPAA follows the data everywhere. Training, vendor sharing, and cloud inferenceinferenceThe moment a trained AI model is put to work: it takes a new input and produces an answer, prediction or generated output.View full definition → all need de-identification or a signed BAA with encryption.
- ONC HTI-1 is a label, not an approval. Ensure the model's source attributes are visible inside certified EHRs.
- The Joint Commission ties it together. Post-deployment monitoring, bias checks, and staff disclosure are your responsibility regardless of who built the model.