HIPAA compliant AI is not a product label, because no agency certifies AI tools for HIPAA. Using AI on patient documents is possible when the provider signs a business associate agreement, the contract limits what it may do with your data, including model training, and your Security Rule safeguards cover prompts, outputs and logs.
What HIPAA compliant AI means in practice
HIPAA compliant AI means an AI service used in a way that meets the HIPAA Rules: under a business associate agreement, with safeguards on every copy of protected health information (PHI) it touches, and inside your own risk analysis. The phrase describes a setup and a contract, not a property of the model.
HHS does not hand out approvals. In its guidance on HIPAA and cloud computing, asked which providers offer HIPAA-compliant services, the HHS Office for Civil Rights answers: "OCR does not endorse, certify, or recommend specific technology or products." An AI service hosted by an outside company is a cloud service in this sense. The same guidance says a provider that processes or stores electronic PHI on your behalf is a business associate, and that using one without a business associate agreement puts the covered entity "in violation of the HIPAA Rules."
The Security Rule then sets the safeguards, and it is deliberately technology neutral. The HHS summary of the HIPAA Security Rule says the rule "does not dictate the specific security measures that a regulated entity must use." Instead, 45 CFR 164.306 on the eCFR requires covered entities and business associates to protect the confidentiality, integrity and availability of electronic PHI and to "protect against any reasonably anticipated threats or hazards."
So a practical definition has three parts:
- A signed agreement. The AI provider is your business associate, or your vendor's subcontractor, under a written contract.
- Limits on use. The contract says what the provider may do with prompts and outputs, and what it may not.
- Safeguards. Logins, activity records, transmission protection and deletion apply to the AI step like any other system holding PHI.
Encryption is one of those safeguards, and a common question when documents are sent to a model. The guide to encryption at rest and in transit for PHI covers what the rule asks for and how to check it.
Can you paste PHI into a general AI chatbot?
Using general AI chatbots with PHI is ruled out unless the chatbot's provider has signed a business associate agreement with your organization and the account you use is covered by it. A personal or consumer sign-up accepts the provider's standard terms, which are not a business associate agreement. Pasting a patient's EOB into such an account discloses PHI.
The reasoning follows from the business associate rules. 45 CFR 164.502(e) on the eCFR lets a covered entity disclose PHI to a business associate only if it "obtains satisfactory assurance that the business associate will appropriately safeguard the information," and those assurances must be documented in a written contract. A chatbot that receives a pasted page, processes it and returns an answer is creating, receiving and maintaining PHI. Without the contract, the assurance does not exist.
It helps to picture the everyday case. A biller is stuck on a denied claim. She copies the EOB text, including the example patient name Dana Ruiz, member ID W123456789 and the remark codes, into a free chatbot and asks what the codes mean. The question is reasonable. The disclosure is the problem. The same question with the codes alone, such as "what does CO-45 mean on an EOB?", contains no PHI and needs no agreement.
Some organizations use enterprise AI accounts that are covered by a business associate agreement. The test is the signed agreement and the account it covers, not the brand or the price tier. A covered enterprise workspace and an uncovered personal account from the same provider can look identical on screen.
Practical controls:
- Publish a list of approved AI tools and the accounts they cover.
- Name general chatbots in security training as a form of disclosure.
- Give staff an approved way to ask code and policy questions without patient details.
- Treat a screenshot pasted into a chatbot the same as typed text.
Does the AI vendor train models on your PHI?
PHI in AI training data is a contract question first. A business associate may use PHI only as its business associate agreement permits or as law requires. If the agreement does not allow training on your documents, that use is outside the contract. Get the answer in writing, not from a settings page.
The rule text sets the frame. 45 CFR 164.504(e) on the eCFR says the contract must "establish the permitted and required uses and disclosures" of PHI by the business associate, and must provide that the business associate will "not use or further disclose the information other than as permitted or required by the contract or as required by law." Under 164.502(a)(3), a business associate "may use or disclose protected health information only as permitted or required by its business associate contract."
The same section allows a contract to permit some uses for the business associate's own "proper management and administration." Whether a vendor reads that to include model improvement is exactly the kind of ambiguity to remove. Ask for a clause that says, in plain words, whether your PHI or anything derived from it may be used to train, fine-tune or evaluate models, and for whose benefit.
Training is one of three related questions:
- Training. May your documents, prompts or outputs be used to improve any model, including ones other customers use?
- Retention. How long are prompts and outputs kept, and where? Do they live in backups after deletion?
- Human review. May the vendor's staff read prompts, for example to investigate abuse or debug a failure? Who are they, and are they bound by the agreement?
The end of the contract matters too. 164.504(e) requires the business associate, at termination, to "return or destroy all protected health information" it still holds, if feasible. A model already trained on your PHI cannot easily give it back, which is one more reason to settle training before any real document is sent.
Questions for an AI model provider that handles PHI
The questions for an AI model provider that handles PHI cover its contract, its logging, the region where data is processed, the subprocessors behind it, and whether it keeps inputs at all. The goal is to know every place a prompt or document goes, who can see it, and how long it stays.
The table below compares three common setups. The rows describe what to confirm for each, not what any particular provider does.
| Question | Consumer chatbot account | Enterprise AI service under a BAA | Self-hosted model |
|---|---|---|---|
| BAA available? | Not through a consumer sign-up; standard terms are not a BAA | Yes, if signed and the account is covered | No AI vendor is involved; your hosting provider needs one |
| Training on inputs | Set by consumer terms; you cannot negotiate it | Settled in the contract; ask for a written no-training clause | You decide |
| Prompt retention | Set by consumer terms | Agreed in the contract, including any zero-retention option | Your own retention policy |
| Logging | Provider's logs, outside your control | Agreed logs, available to you | Your logs, which hold PHI and need protection |
| Who can see inputs | Unknown to you | Named staff and subprocessors, bound by the BAA | Your workforce and your hosting provider |
Questions to put to an enterprise provider:
- Will you sign a business associate agreement, and which accounts and features does it cover?
- Are prompts, documents and outputs used for training or evaluation, for us or anyone else?
- How long are prompts and outputs kept, including in logs and backups?
- Is a zero-retention mode available, and what does it exclude?
- In which region are prompts processed and stored?
- Which subprocessors handle prompts, and are they under agreements of their own?
- Who on your staff can read prompts, and when?
- How and when will you report a security incident to us?
Question 8 has a rule behind it. 45 CFR 164.314 on the eCFR requires the agreement to make the business associate "report to the covered entity any security incident of which it becomes aware," and to flow Security Rule duties down to subcontractors. Question 6 opens a longer topic, covered in the section on where a vendor's subprocessors touch PHI.
Region has an HHS answer as well. The cloud guidance says storing PHI outside the United States is allowed under a business associate agreement, but that "the risks to such ePHI may vary greatly depending on its geographic location," and those risks belong in the risk analysis.
When the AI step is a vendor's tool rather than a raw model, ask the same questions of the vendor and of the model provider behind it. Summarizing long records is a common use; the guide to medical record summarization covers what that task involves. Where the summaries and source files are stored afterward is a records question, covered in HIPAA compliant document management. Schema Care belongs on the vendor list for this review like any other tool that touches patient pages: a business associate agreement comes with each paid plan, and the compliance section of the homepage sets out what has to be in place before patient data arrives.
HIPAA compliant AI edge cases: de-identified prompts and self-hosted models
Two HIPAA compliant AI edge cases change the rules: prompts that contain only properly de-identified data, and models you run on your own infrastructure. De-identified data is not PHI, so an AI service that sees only that data is not a business associate. A self-hosted model removes the AI vendor, but not your own Security Rule duties.
De-identified prompts. The HHS cloud guidance says a provider "is not a business associate if it receives and maintains" only information de-identified under the Privacy Rule. The standard is strict. 45 CFR 164.514 on the eCFR requires either an expert determination that the risk of identifying someone "is very small" or the removal of a defined list of identifiers. Deleting the name from a prompt but keeping the date of service, ZIP code and claim number is not de-identification. A safe workflow strips identifiers before the text reaches the model, and the steps for doing that across a document set are in PHI de-identification in five steps.
Self-hosted models. Running an open model on servers you control means no AI company receives your prompts. Your organization still holds every Security Rule duty for that system: unique logins, activity records, protection in transit and disposal of old data. If the servers sit with a hosting provider, that provider maintains your PHI and needs a business associate agreement, even if it never looks at the data. HHS says so directly for providers holding encrypted data without the key.
Outputs. A summary, a list of extracted fields or an answer generated from a patient document is still PHI when it identifies the patient. The model's output needs the same handling as its input.
Testing. Trying a new AI tool on real charts "just to see" is a disclosure. Test on de-identified or fictional samples until the agreement is signed.