HIPAA-Compliant AI Chatbots: What to Check Before You Buy
“HIPAA compliant” is one of the most overused phrases in chatbot marketing. It gets printed on pricing pages by vendors who have never signed a Business Associate Agreement, and it gets assumed by practices who installed a chat widget without asking where the conversations end up.
This is a plain-English buyer’s guide: what HIPAA actually requires of a chat tool, when a website conversation turns into protected health information, what a BAA has to cover, the technical safeguards to verify, how ChatGPT and Claude fit in, and — just as important — when you should not put an AI agent on patient data at all.
One disclaimer up front. This is not legal advice. HIPAA interpretation is fact-specific, and your compliance counsel has the final word. What this guide gives you is the vocabulary and the questions, so that conversation is shorter and sharper.
What HIPAA actually covers (and what it doesn’t)
HIPAA — the Health Insurance Portability and Accountability Act of 1996 — regulates how covered entities and their business associates handle protected health information (PHI). Three terms, one definition each:
- Covered entity— health plans, healthcare clearinghouses, and healthcare providers that transmit health information electronically in connection with standard transactions such as insurance claims. Most dentists, physicians, and chiropractors who bill insurance fall in this group.
- Business associate— a vendor that creates, receives, maintains, or transmits PHI on behalf of a covered entity. A chatbot vendor that receives patient conversations lands here.
- PHI— individually identifiable health information. It includes the obvious (diagnoses, treatment, medications) and identifiers like names, phone numbers, email addresses, dates, and IP addresses when they are linked to a person’s health, care, or payment for care.
The HHS HIPAA rules that matter most for chat tools are the Privacy Rule (how PHI may be used and disclosed, including the minimum necessary principle) and the Security Rule (administrative, physical, and technical safeguards for electronic PHI).
What federal HIPAA usually does notcover: veterinary practices, and businesses that are not covered entities — for example some cash-only aesthetic or wellness services. That does not make health data a free-for-all: state laws such as Washington’s My Health My Data Act, the FTC, and patient trust all still apply. Whether your business is a covered entity is a question for counsel, not for a vendor’s sales page.
When a chat conversation becomes PHI
The honest answer is that it depends on what the chat touches — and that you do not fully control it, because patients type whatever they want.
A dental chat that asks “what brings you in?” will get answers like “tooth pain since the crown I had placed last year.” That is PHI the moment it is typed. Designing a flow that never asksfor health details helps, but it does not stop patients from volunteering them. The compliance question is not “will PHI arrive?” — it is “what happens to it when it does?”
The common healthcare chat scenarios, roughly in order of risk:
- General questions only (hours, services, directions). Lowest risk, but free-text input still means PHI can show up.
- Appointment requests.Name, contact details, and a booking at a healthcare practice — treat as PHI if you are a covered entity.
- Pre-visit screening or symptom triage.Clearly PHI — BAA and full Security Rule safeguards required.
- Insurance verification. Carrier and member details are PHI; add PCI-DSS if the same flow takes card payments.
- Post-visit follow-up or treatment questions. The conversation is effectively part of the medical record. This belongs in an authenticated patient portal, not a public website chat.
For how booking flows work in general, see our AI appointment booking guide — and apply everything on this page before any of it touches patient data.
The Business Associate Agreement: what it is and what it must cover
A BAA is the contract that lets a vendor lawfully handle PHI on a covered entity’s behalf. It obligates the vendor to safeguard the data, limit its use, report breaches, flow the same obligations down to its own subcontractors, and return or destroy PHI when the contract ends. HHS publishes sample BAA provisions that show the required clauses.
The part buyers miss is the chain. A chatbot vendor typically sends your conversations through a hosting provider, an AI model provider, a database, and often logging, analytics, and email services. Each of those that touches PHI needs to be covered. A vendor that signs a BAA with you but pipes transcripts through an uncovered analytics tool has a gap you inherit.
A note on our own setup, because it illustrates the point: Zellyfi works with Anthropic for its AI models. Working with a model provider that offers BAAs to eligible customers does notmake a chatbot vendor HIPAA compliant. Zellyfi does not sign BAAs with customers and is not built to hold PHI — so we are not the right tool for any of the PHI scenarios above.
The 7 safeguards to verify before a chatbot touches PHI
If a vendor cannot answer “yes, and here is our documentation” to each of these, they are not ready for PHI. Most map to the HIPAA Security Rule’s technical safeguards (45 CFR 164.312); the rest are operational expectations that have become standard in vendor reviews.
1. A signed BAA — and a list of subcontractors it covers
Not “we are HIPAA compliant” on a web page: a signed agreement, plus a written list of every subprocessor that will see your data and confirmation each is under its own BAA.
2. Encryption in transit
Every hop — browser to vendor, and vendor to its own services and model provider — over modern TLS. HIPAA treats encryption as an “addressable” specification, which means a vendor that skips it must document an equivalent alternative. In practice, you should expect encryption.
3. Encryption at rest — everywhere the data lands
Databases are usually encrypted by default on modern cloud platforms. Log aggregation, backups, analytics warehouses, and exported transcripts often are not. Ask where each copy of a conversation lives.
4. Access controls and audit logs
Role-based access, unique user IDs, and an audit trail that can answer “who viewed this patient’s chat, and when?” A generic application log of HTTP requests is not an audit trail. Ask for a sample entry.
5. Retention limits and secure deletion
A documented retention period for transcripts, a deletion process at the end of it, and confirmation that backups and analytics copies follow the same schedule. “We keep everything to improve the product” is a red flag.
6. Breach notification terms
HIPAA requires a business associate to notify the covered entity of a breach without unreasonable delay and no later than 60 days after discovery. Many BAAs negotiate a shorter window. Ask what the contract says and what the vendor’s incident response process looks like.
7. Minimum-necessary configuration of the AI itself
The agent should be configured not to ask for health details it does not need, and to route clinical questions to a human. Ask the vendor to walk you through what the agent is designed never to collect, and how it responds when a patient volunteers it anyway.
Common failure patterns
The general-purpose chat widget on a healthcare site
A practice installs a standard chat or AI widget on its website on a plan that comes with no BAA. Patients start describing symptoms. Every one of those transcripts now sits with a vendor that has no obligation to protect it as PHI.
The BAA with holes in the chain
The vendor signs a BAA, but transcripts also flow to an uncovered analytics tool, translation API, or model provider. The paperwork looks right; the data flow does not.
Training on customer conversations
Transcripts are used to tune a model or prompts shared across customers. That is a use of PHI the BAA almost certainly does not permit. “Do you train on our data?” belongs on every questionnaire.
Retention drift
The policy says 12 months; backups and analytics copies keep conversations for years. Regulators look at where the data actually is, not at the policy document.
Marketing claims that outrun reality
“HIPAA-ready” or “HIPAA-compliant” badges from vendors that will not sign a BAA. Under HIPAA, a business associate without a BAA is not compliant, whatever the badge says — and deceptive privacy claims can also draw FTC attention.
When NOT to put an AI agent on PHI
Sometimes the right answer is not a better vendor — it is keeping AI out of the flow entirely, or keeping PHI out of the AI. Skip the AI agent (or redesign the flow) when:
- The vendor will not sign a BAA. Then it must not receive PHI. That includes Zellyfi.
- The task is clinical. Symptom triage, medication or refill questions, test results, and post-treatment advice need licensed staff and an authenticated channel, not a public website chat.
- The data is especially sensitive. Mental health, substance use, reproductive and sexual health, and information about minors carry extra legal protection under federal and state law.
- You cannot keep PHI out of free-text fields.If the flow invites “tell us what’s going on,” PHI will arrive. Either move that step into your practice management system’s patient portal (already covered by your existing BAA) or use a vendor that is covered.
- You cannot answer the vendor questionnaire below. If you do not know where transcripts go, you cannot assess the risk.
A pattern that works for many practices: keep the public website chat to non-clinical questions, show a short notice asking visitors not to share medical details there, and hand anything clinical to the phone or a patient portal that is already under a BAA.
The vendor questionnaire: 12 questions to ask before signing
Send these to every AI chatbot vendor you evaluate. Any vendor that cannot answer in writing is not ready for patient data.
- Do you sign a Business Associate Agreement? On which plans?
- Which subcontractors touch our data (hosting, AI model provider, database, logging, analytics, email), and is each covered by a BAA with you?
- What encryption do you use in transit and at rest, including backups and logs?
- Do you use customer conversations to train or tune models or prompts? How is that prevented technically?
- Can you show a sample audit-log entry for access to a conversation?
- What is the retention period for transcripts, and how is deletion verified?
- What breach-notification timeline will the BAA commit to?
- Which of your staff can access customer conversations, and under what controls?
- Is our data logically separated from other customers’ data?
- How is the AI prevented from requesting health information it does not need?
- Do you have a SOC 2 Type II report, HITRUST certification, or equivalent you can share under NDA?
- What happens to our data when we cancel, and on what timeline?
If you are comparing tools for a dental practice, our dental AI chatbot comparisoncovers features and pricing — run every option on that list through these 12 questions before any patient data is involved.
Is ChatGPT HIPAA compliant?
The consumer ChatGPT app is not covered by a BAA. Pasting a patient note into it to summarize is not a HIPAA-compliant use.
OpenAI says it can sign BAAs for eligible API customers and for certain enterprise agreements. If a chatbot vendor uses the OpenAI API under a BAA, that covers OpenAI’s link in the chain — the vendor still needs its own BAA with you and its own safeguards. Ask for written confirmation of both.
Is Claude (Anthropic) HIPAA compliant?
Consumer Claude.ai plans are not covered by a BAA.Anthropic offers BAAs to eligible commercial API customers. As with OpenAI, the model provider’s BAA is one link: a chatbot built on the API is only appropriate for PHI if the vendor also signs a BAA with you and implements the safeguards above.
A 30-day plan for choosing a HIPAA-covered chat vendor
Week 1 — Scope
- Decide what the chat must do: general questions, booking, triage, insurance, follow-up.
- List every data field it will collect, and what patients are likely to volunteer.
- Decide which steps should not be in a public chat at all.
- Send the 12-question questionnaire to a shortlist of vendors.
Week 2 — Evaluate
- Cut any vendor that will not sign a BAA or cannot name its subprocessors.
- Have counsel review draft BAAs: breach timeline, subcontractors, permitted uses, return/destruction of data.
- Ask each finalist for references from practices like yours.
Week 3 — Configure
- Sign the contract and BAA, and store the signed copy where you can find it in three years.
- Configure scope, guardrails, and the hand-off to staff for anything clinical.
- Test realistic conversations, including patients volunteering health details.
- Update your risk assessment and privacy notice; train front-desk staff.
Week 4 — Launch carefully
- Start on one page, review early conversations, and tune.
- Confirm you can get access logs from the vendor on request.
- Re-check the BAA scope whenever you add a new feature or workflow.
What a covered chat vendor costs
Pricing varies too much by vendor and contract to quote honestly here, but the structure is predictable, and it is worth knowing before you compare quotes:
| Vendor type | BAA availability | What to check |
|---|---|---|
| General-purpose chat / AI widget | Often none, or only on top tiers | Which plan includes the BAA, and whether AI features are covered by it |
| Enterprise customer-service platform | Sometimes, usually on enterprise contracts | Subprocessor list, AI add-ons, retention defaults |
| Healthcare-specific vendor | Usually standard | The full chain, training on data, audit logs, PMS integration |
| Zellyfi | Not offered | Not suitable for PHI — use only where no health data is involved |
Two rules of thumb. If BAA coverage is gated behind a much higher tier, budget for that tier — the cheaper plan is not an option for PHI. And compare quotes on the same scope: a vendor covering booking only is not comparable to one covering triage and insurance.
Red flags that should end the evaluation
- “We’re HIPAA compliant” — but no BAA. For a business associate, there is no compliance without the agreement.
- The vendor cannot name its AI model provider or confirm that link in the chain is covered.
- The demo shows real patient conversations from other customers.
- “HIPAA doesn’t apply to chat widgets.” It applies to PHI, wherever it is handled.
- Vague answers on training, retention, or access to more than a couple of the questionnaire items.
What auditors ask for
The HHS Office for Civil Rights (OCR) enforces HIPAA, most often through investigations triggered by a breach or a complaint. If a chat tool is involved, have these ready:
- The signed BAA with the chat vendor — the signed copy, not a template.
- Confirmation that the vendor’s subcontractors are covered.
- Your risk assessment covering the chat tool, updated when its scope changed.
- Your privacy notice and staff training records.
- Access logs for your data, obtainable from the vendor on request.
The OCR breach portal lists reported breaches affecting 500 or more individuals and is a useful way to see which failure patterns actually come up.
Adjacent rules that also matter
- State health-privacy laws.Several states regulate consumer health data beyond HIPAA — Washington’s My Health My Data Act is the best-known example — and some apply even to businesses HIPAA does not cover.
- PCI-DSS. If a chat flow takes card details, payment-card rules apply too. Most tools hand payment off to a processor so they never touch card data.
- FTC.The FTC has acted against health-adjacent companies over deceptive privacy claims and sharing health data with advertisers — independent of HIPAA.
For a deeper academic treatment, see “AI Chatbots and Challenges of HIPAA Compliance for AI Developers and Vendors” on NIH PMC.
Frequently asked questions
Is Zellyfi HIPAA compliant?
No. Zellyfi is not HIPAA compliant and does not sign Business Associate Agreements (BAAs). Our AI agent must not be used to collect or store protected health information (PHI). If your website chat needs to handle PHI, choose a vendor that will sign a BAA and can document the safeguards in this guide.
What makes an AI chatbot HIPAA compliant?
No chatbot is compliant on its own — compliance is about how PHI is handled across the whole chain. If the chatbot will create, receive, store, or transmit PHI for a covered entity, the vendor is a business associate: it must sign a BAA and implement the HIPAA Security Rule safeguards (access controls, audit controls, integrity protections, authentication, and transmission security), and every subcontractor that touches the data — hosting, the AI model provider, logging — must be covered too.
Is ChatGPT HIPAA compliant?
The consumer ChatGPT app is not covered by a BAA, so pasting patient information into it is not a HIPAA-compliant use. OpenAI says it can sign BAAs for eligible API customers and for certain enterprise agreements. Even then, the BAA only covers OpenAI's part of the chain — the chatbot vendor and your own practice still have to meet their obligations.
Is Claude (Anthropic) HIPAA compliant?
The same structure applies. Consumer Claude.ai plans are not covered by a BAA. Anthropic offers BAAs to eligible commercial API customers. A model provider's BAA does not make a chatbot vendor compliant by itself — the vendor needs its own BAA with you and its own safeguards.
Do I need a BAA if my chatbot only books appointments?
Often, yes. A name and phone number combined with the fact that someone is booking at a healthcare practice can be PHI, and patients routinely volunteer symptoms or treatment details in free-text chat. If you are a covered entity and the chatbot vendor will receive that data, you need a BAA. If a vendor will not sign one, keep it off any page or flow where patients share health details.
What happens if my AI chatbot violates HIPAA?
HHS can impose civil penalties tiered by culpability and adjusted for inflation every year — from roughly $140 per violation at the lowest tier up to an annual cap above $2 million per identical provision at the highest. A breach can also trigger notification duties to affected patients, HHS, and in larger breaches the media, plus exposure under state law.
The bottom line
A HIPAA-appropriate AI chatbot is not a feature you can see in a demo. It is a signed BAA, a fully covered chain of subcontractors, documented safeguards, and a flow designed so the AI only touches the data it needs. Vendors that have done the work can show you all of it in writing.
Zellyfi has not, and does not claim to: we do not sign BAAs and our AI agent is not for PHI. Where we fit is the large set of businesses whose website conversations do not involve health information — contractors, home services, and professional services. If that is you, our services page explains how the agent works and our pricing page lists every plan. If your chat needs to handle PHI, take the questionnaire above to vendors that will sign a BAA.
No Health Data Involved? Let’s Talk.
Zellyfi builds custom AI agents that answer questions, qualify leads, and book appointments around the clock — for businesses whose conversations do not include protected health information. We do not sign BAAs, so we will tell you plainly if we are the wrong fit.
See How It WorksView Pricing →
