AI Struggles with Basic Data Tasks for Hospital Administrators: Study

Why It Matters

Hospital administrators depend on precise, rapid data aggregation for resource planning; current LLMs cannot be trusted without code‑execution support, limiting immediate AI adoption in clinical operations.

Key Takeaways

• Nine LLMs failed basic patient count prompts on real ED data
• Chain‑of‑thought reasoning gave slight accuracy gains, but not scalable
• Code‑generation tool approach markedly improved results for top models
• Study warns LLMs unsuitable for standalone admin tasks without tooling

Pulse Analysis

The promise of generative AI has sparked interest across health systems eager to automate routine reporting, yet the technology’s readiness for core administrative functions remains uncertain. Hospital leaders often task AI with aggregating admission numbers, bed utilization, and other key performance indicators—processes traditionally handled by business intelligence tools. Early enthusiasm assumed that large language models (LLMs) could replace manual queries, but the Mount Sinai‑Mayo study underscores a gap between hype and practical reliability.

Researchers fed nine leading LLMs real‑world emergency department data from 50,000 visits and asked them to perform two straightforward tasks: tally patients meeting a condition and filter records by multiple criteria. Plain prompts produced erratic counts, while a chain‑of‑thought approach—asking the model to explain its reasoning—offered only marginal gains and faltered as data volume grew. The breakthrough emerged when models were instructed to generate executable code, effectively turning the LLM into a programmer that could query the dataset directly. This tool‑based method restored accuracy, but only for the most advanced models, highlighting a hybrid solution rather than a pure‑LLM answer.

For health‑care executives, the findings signal that AI deployments must incorporate robust tooling and validation layers before replacing existing analytics pipelines. Integrating LLMs with code execution engines or specialized data‑access APIs can deliver the speed of language models while preserving the precision required for resource allocation and compliance reporting. As vendors refine agentic AI architectures, hospitals should pilot these hybrid workflows in low‑risk settings, establish clear performance benchmarks, and retain human oversight to ensure data integrity and patient safety.