AI agent governance in healthcare
Healthcare agent governance turns on one boundary: administrative versus clinical. Cross it and an entirely different regulatory regime applies. Most enterprise healthcare agents are administrative — the governance work is proving they stayed that way.
Get an Agent Trust Gap BriefThe regulatory surfaces
| Surface | What it governs | Bearing on agents |
|---|---|---|
| HIPAA | PHI privacy and security | Minimum necessary access, audit controls, and business associate obligations for any vendor touching PHI. |
| FDA device regulation | Software as a medical device | Clinical decision support can fall in scope depending on function and the clinician's ability to review the basis. Administrative automation generally does not. |
| State AI laws | Varies by state | Consequential-decision duties reach healthcare where agents influence coverage, access or care decisions. |
| Health system procurement | Contractual | Frequently stricter than regulation. Security review is where deals actually slow down. |
| 42 CFR Part 2 | Substance use disorder records | Heightened restrictions beyond HIPAA. Easy to miss when scoping data access. |
Verified 2026-08-07. Not legal advice. Confirm against primary sources and counsel for your specific deployment.
Keeping the boundary provable
The boundary is not maintained by intent, it is maintained by a gate. A system prompt saying "do not provide medical advice" produces no evidence and fails silently. A classification gate on outbound content that blocks and records produces a countable set of refusals — which is what a compliance officer can actually rely on.
- Enumerate permitted actions. Scheduling, eligibility, records retrieval, prior authorisation paperwork, reminders. Anything not listed is refused by default rather than allowed by omission.
- Enforce minimum necessary structurally. Credentials scoped to the encounter, so over-access is impossible rather than discouraged.
- Gate outbound content by channel. What may appear in an SMS is not what may appear in a portal message.
- Produce refusal receipts. The evidence the boundary holds is the record of it holding.
- Retain by obligation. Match the longest applicable retention, not the logging tier's default.
Workload-level detail on scheduling, prior authorisation and patient messaging is in healthcare agent governance.
Common questions
Does HIPAA prohibit AI agents?
No. It sets conditions: minimum necessary access, audit controls, and business associate agreements with vendors handling PHI. Administrative agents are deployable within those conditions.
When does an AI agent become a medical device?
When its function crosses into clinical decision support in ways FDA frameworks reach — which depends on function and on whether a clinician can independently review the basis for the recommendation. Administrative automation generally does not.
How do you prove an agent never gave medical advice?
Refusal receipts from an enforced classification gate. A countable record of blocks is evidence; an absence of complaints is not.
Is a BAA needed with an AI vendor?
If the vendor creates, receives, maintains or transmits PHI on your behalf, yes. This is one of the first questions any health system security review asks.
What slows healthcare AI deals down most?
Security review, not regulation. Health systems ask for evidence of access controls, audit trails and data handling — having those artifacts ready shortens the cycle more than any other single thing.