Tom Griffiths | Mapping The Jagged Edges Of AI With The Tools Of Cognitive Science

Why It Matters

Understanding the jagged AI frontier lets companies deploy language models where they excel and avoid costly failures in domains where models lag, improving risk management and product effectiveness.

Key Takeaways

• AI intelligence is a jagged frontier, not a single dimension.
• Cognitive science tools can map human‑AI capability gaps.
• Similarity judgments reveal shared representations between LLMs and humans.
• LLMs excel at color, pitch, but lag on taste and instrument sounds.
• Multi‑dimensional scaling helps evaluate alignment for specific application domains.

Summary

Tom Griffiths frames the current AI landscape as a "jagged frontier" rather than a linear hierarchy of intelligence, contrasting the historic "great chain of being" with a modern view that organisms—and AI systems—occupy diverse, niche‑specific dimensions. He argues that large language models (LLMs) illustrate this frontier: they outperform humans on some tasks while falling short on others, creating a patchwork of strengths and weaknesses. Griffiths highlights the opacity of LLMs—complex neural architectures, proprietary training data, and inaccessible internal activations—making capability assessment a challenge for computer scientists. He proposes borrowing tools from cognitive science, a field accustomed to studying opaque minds, to map these AI‑human boundaries. Three core methods are discussed: measuring similarity, analyzing categorization strategies, and applying rational analysis. Using similarity judgments, researchers can construct matrices that, after multidimensional scaling, reveal representational spaces such as the human color wheel or the pitch helix. When LLMs are asked to rate similarity of colors or musical notes, they generate comparable structures, indicating alignment in those domains. However, for domains like taste or instrument timbre, the correlation drops, exposing gaps in the models' internal representations. The implication for businesses is clear: cognitive‑science techniques provide a systematic way to evaluate where AI can be trusted and where human expertise remains essential. By quantifying alignment across task dimensions, firms can better allocate AI resources, mitigate risks, and design products that leverage model strengths while compensating for their blind spots.

Original Description

Subscribe to Foresight Secure AI Seminar Group: https://foresight.org/seminar/secure-ai-seminar-group/

Supporting researchers, engineers, and entrepreneurs in computer science, ML, crypto, security, and related fields who leverage those technologies to improve cooperation across humans and ultimately AIs.

Tom Griffiths | Mapping The Jagged Edges Of AI With The Tools Of Cognitive Science

Abstract: Current artificial intelligence systems demonstrate surprising amount of heterogeneity in their abilities, displaying superhuman competence in some tasks but puzzling limitations in others. I will argue that the tools we need for understanding this heterogeneity can be found in cognitive science, where researchers have spent decades developing theoretical and empirical methods for making sense of the capabilities of intelligent systems. Work by cognitive scientists suggests two strategies for mapping the jagged edges of AI: identifying general properties of neural networks that might translate into limitations for current AI systems, and considering cases where human minds might provide a guide to problems that pose a challenge for AI. I will present examples of both strategies, discussing some surprising cases where large language models perform poorly in predictable ways and recent results using the limits of human cognition to predict cases where large language models and vision language models fail.

Speaker Bio: Tom Griffiths is the Henry R. Luce Professor of Information Technology, Consciousness and Culture in the Departments of Psychology and Computer Science at Princeton University. He is also the Director of the new Princeton Laboratory for Artificial Intelligence, which focuses on identifying areas where AI can have transformative impact on research. His work explores connections between human and machine learning, using ideas from statistics and artificial intelligence to understand how people solve the challenging computational problems they encounter in everyday life. He has made contributions to the development of Bayesian models of cognition, probabilistic machine learning, nonparametric Bayesian statistics, and models of cultural evolution, and his recent efforts have explored how methods from cognitive science can shed light on modern artificial intelligence systems. Tom completed his PhD in Psychology at Stanford University in 2005, and taught at Brown University and the University of California, Berkeley before moving to Princeton. He has received awards for his research from organizations ranging from the American Psychological Association to the National Academy of Sciences and is a co-author of the book Algorithms to Live By, introducing ideas from computer science and cognitive science to a general audience. His new book The Laws of Thought tells the story of the quest to find a mathematical theory of the mind and was published in February 2026.

══════════════════════════════════════

About The Foresight Institute

The Foresight Institute is a research organization and non-profit that supports the beneficial development of high-impact technologies. Since our founding in 1986 on a vision of guiding powerful technologies, we have continued to evolve into a many-armed organization that focuses on several fields of science and technology that are too ambitious for legacy institutions to support. From molecular nanotechnology, to brain-computer interfaces, space exploration, cryptocommerce, and AI, Foresight gathers leading minds to advance research and accelerate progress toward flourishing futures.

We are entirely funded by your donations. If you enjoy what we do please consider donating through our donation page: https://foresight.org/donate/

Visit https://foresight.org, subscribe to our channel for more videos or join us here:

• Twitter: https://x.com/foresightinst

• Facebook: https://www.facebook.com/foresightinst

• LinkedIn: https://www.linkedin.com/company/foresight-institute

══════════════════════════════════════

Timecodes

00:00 Intro

00:55 Great Chain Of Intelligence

03:20 Mapping Jagged AI

06:20 Cognitive Science Tools

06:50 Similarity And Representation

11:30 Number Representations

15:50 Category Boundary Failures

21:50 Rational Analysis

23:30 Embers Of Autoregression

33:40 Reasoning Can Hurt

37:10 Human AI Complementarity

39:20 Q&A