The Verification Imperative

In 2023, two New York lawyers were sanctioned for filing a brief full of cases that did not exist. Their AI had invented every one - complete with realistic citations, quotations, and docket numbers. They are no longer alone.

The court decisions sanctioning lawyers for AI-fabricated citations now number in the thousands, and grow by several a day. The tools that caused them are marketed as "trustworthy," "verified," and "hallucination-free" - yet when Stanford researchers tested the leading purpose-built legal AI products, they still produced false information on as many as one in three queries.

So how is a working lawyer supposed to know which tools to trust, and how far?

The Verification Imperative answers that question with a framework, not a sales pitch. Drawing on the IEEE's recognized criteria for trustworthy AI - accountability, transparency, privacy, and bias - it gives lawyers a durable lens for evaluating any legal-AI tool, including ones that don't exist yet. It is built around five questions you can put to any vendor before you let their tool near a client's matter:

• Show me how I verify this.
• Where does my data go?
• What can't this tool see?
• Who is responsible when it's wrong?
• And how does it fail - loudly and safely, or silently and catastrophically?

To show what those questions reveal, the author builds and then deliberately breaks a real citation-verification tool - documenting exactly where it worked, and exactly where it failed on a case every first-year law student knows. No tool is sold. No vendor is spared. The reader leaves not with a product to trust, but with a practice: the oldest discipline in law - you do not get to assert what you cannot verify - applied to its newest and most convincing instrument.

Because the responsibility never transfers to the machine. It never could.

Written for practicing attorneys, general counsel, and legal professionals navigating the real risks of AI in practice. Not legal advice; a framework for judgment. Written by a holder of the IEEE CertifAIEd™ professional certification, who built the tool he dissects.