Hallucinations” By West & Lexis AI?

Key Takeaways

• Lexis+ AI hallucinated in 17‑33% of queries, lower than GPT‑4.
• Westlaw AI showed highest hallucination rate, about one‑third of responses.
• Study stresses verification as ethical mandate for attorneys using AI tools.
• Vendors dispute findings; internal metrics claim lower hallucination rates.

Pulse Analysis

The 2025 Stanford‑Yale paper provides the first systematic benchmark of commercial legal research assistants, revealing that even purpose‑built systems are not immune to factual hallucinations. By measuring both accuracy and grounding, the authors showed that Lexis+ AI, while the most reliable at 65% correct answers, still produced false citations in roughly one‑quarter of queries. Westlaw’s AI‑Assisted Research lagged behind, with a one‑third hallucination rate that could mislead practitioners relying on its output for brief preparation or courtroom arguments. These metrics matter because they quantify a risk that was previously anecdotal, giving firms a data‑driven basis for policy decisions.

For law firms, the practical implication is clear: AI tools should be treated as research aides, not authoritative sources. The study’s recommendation to verify every proposition aligns with the American Bar Association’s competence standards, which obligate attorneys to ensure the reliability of information presented to clients and courts. Integrating a verification workflow—cross‑checking citations against primary sources and using secondary validation tools—mitigates the ethical exposure highlighted by the research. Moreover, firms can leverage the comparative performance data to select the tool that best fits their risk tolerance and practice area.

The industry response illustrates the tension between vendor confidence and independent scrutiny. Thomson Reuters and LexisNexis have publicly contested the study’s methodology and cited internal improvements made after the data collection period, including second‑generation releases that claim reduced hallucination rates. Nonetheless, the lack of a publicly available, standardized benchmark leaves practitioners dependent on third‑party evaluations. As AI continues to evolve, ongoing, transparent assessments will be essential to maintain trust and to ensure that the promise of faster, more comprehensive legal research does not come at the cost of accuracy and professional responsibility.