This AI Knew the Answers but Didn’t Understand the Questions

Why It Matters

The findings caution investors and developers that surface performance can mask fundamental gaps in AI reasoning, urging more rigorous evaluation before deploying such models in high‑stakes applications.

Key Takeaways

• Centaur excelled in 160 cognitive tasks per 2025 Nature paper
• Zhejiang study shows model overfits, not truly understanding prompts
• Model defaults to learned answer patterns despite nonsensical instructions
• Findings warn against relying on surface performance metrics alone
• Emphasizes need for deeper language comprehension tests in AI

Pulse Analysis

Centaur’s debut in 2025 generated headlines by claiming to simulate human cognition across a broad suite of psychological tasks. Built on a standard large‑language model and fine‑tuned with experimental data, it appeared to bridge the gap between AI and cognitive science, offering a potential tool for researchers and a glimpse of machines that think like people. The buzz reflected a broader industry trend: leveraging language models to model complex mental processes, from decision‑making to executive control, promising new insights and commercial applications.

The Zhejiang University team, however, exposed a critical flaw. By replacing task‑specific prompts with a simple instruction to "choose option A," they demonstrated that Centaur continued to output the answers it had seen during training, rather than interpreting the new request. This behavior points to classic overfitting: the model learns statistical regularities in its training set without grasping underlying semantics. Such pattern‑matching can produce impressive benchmark scores while failing on out‑of‑distribution queries, a risk that grows as models are deployed in real‑world decision contexts where nuance matters.

For the AI industry, the study is a reminder that evaluation must move beyond headline‑grabbing metrics. Robust, adversarial testing that probes intent, context, and reasoning is essential to distinguish genuine understanding from memorization. As investors fund ever larger models, developers need transparent diagnostics and interdisciplinary collaboration with cognitive scientists to design benchmarks that reflect true language comprehension. Only then can the promise of AI that truly mirrors human thought be realized, reducing hallucinations and building trust in critical applications such as healthcare, finance, and education.