Foresight Institute

Tom Griffiths | Mapping The Jagged Edges Of AI With The Tools Of Cognitive Science

Tom Griffiths frames the current AI landscape as a "jagged frontier" rather than a linear hierarchy of intelligence, contrasting the historic "great chain of being" with a modern view that organisms—and AI systems—occupy diverse, niche‑specific dimensions. He argues that large language models (LLMs) illustrate this frontier: they outperform humans on some tasks while falling short on others, creating a patchwork of strengths and weaknesses. Griffiths highlights the opacity of LLMs—complex neural architectures, proprietary training data, and inaccessible internal activations—making capability assessment a challenge for computer scientists. He proposes borrowing tools from cognitive science, a field accustomed to studying opaque minds, to map these AI‑human boundaries. Three core methods are discussed: measuring similarity, analyzing categorization strategies, and applying rational analysis. Using similarity judgments, researchers can construct matrices that, after multidimensional scaling, reveal representational spaces such as the human color wheel or the pitch helix. When LLMs are asked to rate similarity of colors or musical notes, they generate comparable structures, indicating alignment in those domains. However, for domains like taste or instrument timbre, the correlation drops, exposing gaps in the models' internal representations. The implication for businesses is clear: cognitive‑science techniques provide a systematic way to evaluate where AI can be trusted and where human expertise remains essential. By quantifying alignment across task dimensions, firms can better allocate AI resources, mitigate risks, and design products that leverage model strengths while compensating for their blind spots.