Surprise Upset: GPT-5.5 Beats Claude Fable 5 on Brutal New Agents’ Last Exam Benchmark

Why It Matters

ALE provides enterprises with a realistic gauge of whether AI agents can deliver economically valuable work, reducing reliance on inflated benchmark scores. The results signal that current leading models are still far from reliably handling complex, multi‑step professional tasks.

Key Takeaways

• GPT‑5.5 tops ALE leaderboard with 24% pass rate.
• Claude Fable 5 ranks third, scoring 22% passes.
• ALE evaluates full‑stack agentic workflows across 55 industry domains.
• Only 10% of ALE tasks are public to prevent benchmark contamination.
• Even top models fail hardest ‘Last‑Exam’ tier, achieving 0% passes.

Pulse Analysis

The AI community has long wrestled with benchmarks that reward narrow, synthetic tasks rather than genuine productivity. ALE tackles this by embedding agents in a Generalist Computer‑Use Agent framework that demands visual perception, tool interaction, and runtime management across realistic software stacks such as Siemens NX, Unreal Engine, and Adobe After Effects. By anchoring tasks to the U.S. O*NET occupational taxonomy, the benchmark translates abstract performance numbers into concrete labor relevance, offering a clearer signal for investors and product teams.

OpenAI’s GPT‑5.5 emerging as the ALE leader underscores the company’s edge in handling intricate, multi‑part prompts, a critical advantage for enterprise deployments that require reliable end‑to‑end execution. Anthropic’s Claude Fable 5, despite its hype, lagged behind, exposing weaknesses in maintaining instruction fidelity over long workflows. The stark 0% pass rate on the hardest tier for all major models, including Google’s Gemini CLI, serves as a reality check: current agentic systems still stumble when tasked with high‑stakes, licensed‑software operations that mirror real‑world revenue‑generating activities.

Looking forward, ALE’s “living benchmark” model—keeping 90% of tasks private and rotating them over time—mitigates contamination and ensures future evaluations remain meaningful. For businesses allocating billions to AI agents, the benchmark offers a compass to differentiate fleeting marketing claims from durable, production‑ready capability. As model developers iterate to improve reasoning, perception, and tool use, ALE will likely become a standard yardstick, shaping both R&D roadmaps and procurement decisions across the AI‑driven enterprise landscape.