Can AI Really Simulate Human Thinking? Research Casts Doubt on an Influential Study, Suggesting an Advanced Model Was Just Really Good at Memorizing Patterns.

Why It Matters

If AI performance is driven by pattern memorization, claims of approaching artificial general intelligence may be premature, reshaping research priorities and funding.

Key Takeaways

• Centaur achieved 64% accuracy on 2025 human‑behavior tests.
• New study attributes performance to overfitting, not true understanding.
• Modified prompts reveal model repeats learned answer patterns.
• Findings raise doubts about near‑term artificial general intelligence claims.
• Stress‑testing recommended to separate performance from genuine cognition.

Pulse Analysis

The 2025 Nature article on Centaur sparked excitement by suggesting that a single foundation model could predict human choices across a broad set of experiments. By training on more than 10 million decisions from 60 000 participants, the researchers reported a striking 64% match rate, positioning the model as a potential bridge between computational neuroscience and AI. Media coverage framed the result as a milestone toward artificial general intelligence, prompting investors and labs to double down on large‑scale language models as cognitive simulators.

The follow‑up study published in January 2026 takes a more skeptical stance, focusing on the statistical phenomenon of overfitting. Ding and Liu’s simple yet powerful probe—forcing the model to select a predetermined option—revealed that Centaur continued to produce correct answers, indicating it was echoing memorized patterns rather than reasoning. Their analysis underscores a methodological blind spot: high benchmark scores can mask shallow shortcuts when training and test distributions align. By exposing this gap, the researchers call for rigorous stress‑testing that separates genuine understanding from clever pattern matching, a practice that could become a new standard in AI evaluation.

Beyond the technical dispute, the controversy reverberates through the broader AI community. If current architectures are fundamentally limited by reasoning failures, the timeline for achieving true AGI may need recalibration. Stakeholders—from venture capitalists to policy makers—must recognize that headline metrics alone do not guarantee cognitive depth. Future work will likely emphasize hybrid approaches, incorporating symbolic reasoning or neuro‑symbolic models, and will demand transparent validation pipelines that test models on out‑of‑distribution scenarios. In this evolving landscape, the Centaur debate serves as a cautionary tale: impressive performance must be matched with robust proof of underlying comprehension.