HAI Seminar: Code World Models for General Game Playing

Why It Matters

By giving LLMs the ability to synthesize executable world models, these methods turn language models into toolmakers that enable far more efficient, robust planning in novel or out-of-distribution environments—impacting game AI, simulation-driven decision systems, and any domain where building rapid, testable simulators matters.

Summary

DeepMind researchers presented two approaches for general game playing that use LLMs to synthesize executable game models from natural-language rules and play trajectories. In the “code world model” method, an LLM generates a faithful game implementation (with unit-test-like trajectories) which is then solved at runtime by planners such as Monte Carlo tree search or RL. The team also described a hybrid workflow in which the LLM and synthesized code interact online to choose actions, and showed these model-based strategies outperform vanilla LLM-as-policy and naive code-as-policy baselines across a range of deterministic, stochastic and multiplayer games. Both lines of work have been published (ICLR and workshop) and relax prior constraints like full observability and determinism.

Original Description

While large language models show promise in many domains, relying on them for direct policy generation in games often results in illegal moves and poor strategic play. In this HAI Seminar, DeepMind Research Scientist Wolfgang Lehrach presents a novel approach that uses LLMs to translate natural-language game rules into executable symbolic world models (CWMs). These models enable AI systems to reason about game environments more accurately and support advanced planning through Reinforcement Learning (RL) and Monte Carlo Tree Search (MCTS). The talk explores how combining language models with symbolic reasoning can improve decision-making in both perfect and imperfect information games.

This video was recorded at Stanford University on May 13, 2026.

00:00:00 Presentation

00:35:41 Q&A