Should Legal AI Companies Be Liable for Hallucinations?

Summary of an essay from Dorna Moini's Legal Tech Notes, published on Substack on April 21, 2026. Dorna Moini is the CEO and Founder of Gavel and a former litigator at Sidley Austin. This page gives the argument and the numbers; the full essay is on Substack.

In brief: When attorneys from K&L Gates and Ellis George filed a federal brief with nine botched citations and two nonexistent cases produced with CoCounsel, Westlaw Precision and Google Gemini, the special master struck the brief and imposed $31,100 in sanctions on the firms. Thomson Reuters and Google paid nothing. The essay asks whether vendors should be liable, finds the case stronger than expected, and still answers no: the lawyer's duty to verify is nondelegable, and the better lever is mandatory, standardized disclosure of hallucination rates.

What do pharma, credit ratings and GPS tell us about vendor liability?

Three industries have already answered a version of this question. In pharmaceuticals, the learned intermediary doctrine puts the prescribing doctor between the manufacturer and the patient, so a bad prescribing decision is malpractice against the doctor rather than a product claim against the drug maker. Legal AI has the same structure, with the lawyer as the intermediary; a scalpel maker is not sued when a surgeon slips.

Credit rating agencies gave AAA ratings to mortgage securities that failed in 2008 and escaped liability by calling ratings "opinions" protected by the First Amendment, the same posture legal AI vendors take in their terms of service. Accountability came from reputation, more skeptical institutional buyers, new competitors and tighter SEC disclosure rules. And GPS: drivers have followed directions into lakes and off closed bridges, and courts have rarely held Garmin or Google liable, because the driver must still look at the road. Nobody sues Westlaw when a lawyer misreads a real case; a fabricated case does not change the lawyer's duty to check.

How bad is the technical problem, and can vendors fix it?

General-purpose models can hallucinate on up to 43% of legal queries because they are not trained on comprehensive legal databases. Legal-specific tools do better by pairing a model with real legal content through retrieval-augmented generation, meaning the model is handed the actual sources at the moment of the question. Some vendors have marketed "checks and balances" ensuring answers are grounded in good law while a third of their answers contained a hallucination.

The fixes exist. A vendor can cross-reference every citation against a database of real decisions and run deterministic validation that blocks an output which fails the check. The essay's open question is whether those checks should be legally required or whether reputation and buyer pressure should drive adoption.

Why does the lawyer's existing duty settle the question?

Judge Rankin's order against Morgan & Morgan put it directly: signing a filing means the attorney read it and made a reasonable inquiry into the law, and that duty is nondelegable. Federal Rule of Civil Procedure 11 already covers this, so no AI-specific rule is needed. A supervising partner who signs a brief citing an overruled case is responsible whether the error came from an associate, a paralegal, an outdated treatise or a chatbot.

Shifting responsibility to vendors would create moral hazard: if lawyers believe the vendor pays, they verify less. Morgan & Morgan's own response after its sanctions was a mandatory click-through requiring users to confirm they had independently verified AI output.

What should change instead of creating a new tort?

Disclosure. Companies that make reliability claims should publish their hallucination rates using a standard methodology, the way clinical trial data is treated by the FDA as a public input for informed choice rather than a marketing asset. Today general counsel run their own benchmarks and procurement tests in isolation, with no shared standard, while vendor marketing runs ahead of the evidence. A common, published accuracy standard would hold vendors accountable without weakening the lawyer's duty.

Key numbers and sources from the essay

• K&L Gates and Ellis George brief: nine botched citations, two nonexistent cases, $31,100 in sanctions on the firms; tools used were CoCounsel, Westlaw Precision and Google Gemini.
• General-purpose models: hallucinations on up to 43% of legal queries.
• Some legal-specific tools marketed as grounded in good law produced a hallucination in about a third of answers.
• Federal Rule of Civil Procedure 11 and Judge Rankin's Morgan & Morgan order: the duty to verify is nondelegable.
• Morgan & Morgan's fix after sanctions: a mandatory click-through confirming independent verification of AI output.

What this means for a law firm or legal team

1. Assume the sanction lands on the signing lawyer, because it will. Build verification into the filing workflow rather than relying on the tool's marketing.
2. In procurement, ask vendors for their hallucination rate, the methodology behind it and whether citations are validated against a database of real decisions before output.
3. Prefer tools whose work is grounded in documents you supply. Gavel Exec works from the contract in Microsoft Word and your playbook, and shows the source of each suggested change; Gavel's security and data handling are described on the security page.
4. Adopt a firm policy like Morgan & Morgan's click-through: no AI-assisted filing leaves without a named lawyer confirming each citation.

Frequently asked questions

Can a law firm sue an AI vendor for a hallucinated citation?

In theory, but the essay argues the claim is weak and the policy is wrong. Courts treat the lawyer as the responsible intermediary, as with doctors and drug makers or drivers and GPS, and Rule 11 makes the duty to verify nondelegable.

Who paid the sanctions in the K&L Gates and Ellis George case?

The law firms, $31,100 in total. Thomson Reuters and Google, whose tools produced the fabricated citations, paid nothing.

What accountability does the essay propose for legal AI vendors?

Mandatory, standardized disclosure of hallucination rates, published the way clinical trial data is, so buyers can compare tools on evidence rather than marketing.

Read the full essay: Should Legal AI Companies Be Liable for Hallucinations? on Substack, or subscribe to Dorna Moini's Legal Tech Notes.

Related on gavel.io: Gavel Exec, Gavel security, AI hallucination rates in legal work.