Machine Learning Street Talk

If You Can't See Inside, How Do You Know It's THINKING? [Dr. Jeff Beck]

The conversation centers on what it means for a system to "think" and how to recognize agency when internal computations are hidden. Dr. Jeff Beck argues that an agent is distinguished by having internal states that generate policies over long time scales, rather than being a simple input‑output device. He ties this to geometric deep learning, noting that incorporating physical symmetries improves modeling of the world, but the deeper question remains how to infer agency from observable behavior. Key insights include the need for planning and counterfactual reasoning as hallmarks of genuine agency. Metrics such as transfer entropy can estimate how much information a system integrates over time, offering a quantitative, though non‑normative, gauge of agency. Beck also stresses that physical embodiment matters; a high‑fidelity simulation of a brain may replicate behavior, yet without a material substrate he hesitates to call it an agent. Illustrative examples range from labeling a rock as an agent under a broad definition, to dissecting a chess engine that appears to plan but could be reduced to a sophisticated policy function. The dialogue also touches on energy‑based models, contrasting them with standard feed‑forward networks by highlighting their built‑in inductive priors that constrain input‑output relationships, thereby offering clearer interpretability. The implications are twofold: for AI research, developing metrics that capture planning depth and information integration could refine how we label and evaluate autonomous systems; philosophically, the discussion underscores that agency may be a continuum rather than a binary label, urging practitioners to adopt probabilistic, degree‑based frameworks rather than strict categorical distinctions.