The Demise of the Kernel for AI

Why It Matters

If true, the industry’s competitive edge will move from sole chip innovation to system and software orchestration—shifting investment, talent and leverage toward companies that can solve compilers, runtimes and scale challenges, and altering how costly frontier AI workloads are built and deployed.

Summary

Jay Dwani argues that AI performance bottlenecks have moved beyond individual chips to software, compilers and system-level design, as training now runs across tens of thousands of GPUs and complex heterogeneous datacenter fabrics. He says the traditional kernel—the low-level building block for AI optimization—has become a new assembly language: powerful but too rudimentary for modern scale and diversity. Delivering frontier performance requires co-design across programming models, compilers, runtimes, memory hierarchies, interconnects and packaging, not just a better silicon. Dwani points to Nvidia’s success as rooted in systems engineering rather than chip design alone and forecasts a “post-kernel” shift that will reshape who can compete in AI infrastructure.

Original Description

In this episode of AI With Sally, Lemurian Labs CEO Jay Dawani explains why system-level compilation, not faster chips alone, may determine the future of AI infrastructure.

#sponsored