SaaS News and Headlines

Databricks' OfficeQA Uncovers Disconnect: AI Agents Ace Abstract Tests but Stall at 45% on Enterprise Docs

Databricks introduced OfficeQA, a benchmark that tests AI agents on document‑heavy enterprise tasks using 89,000 pages of U.S. Treasury Bulletins. Tests show top agents such as Claude Opus 4.5 and GPT‑5.1 achieve only 37‑44% accuracy on raw PDFs, rising to 68% and 53% respectively when documents are pre‑parsed. The study highlights parsing, document versioning, and visual chart interpretation as the main sources of error. OfficeQA aims to steer research toward real‑world grounded reasoning rather than abstract academic exams.