Ep 23: Cross-Stack Design and Tooling for Large-Scale Distributed AI Systems with Dr. Tushar Krishna, Georgia Tech

In this episode, Dr. Tushar Krishna traces his evolution from network‑on‑chip research to leading the design of AI accelerators. Early work on multi‑hop NoC architectures at AMD and Intel laid the groundwork for understanding wire delays and traffic patterns. When the ImageNet breakthrough arrived, Krishna leveraged that knowledge to build domain‑specific chips like Iris, optimizing data flow for convolutional networks and demonstrating how precise workload mapping can dictate interconnect design.

Krishna emphasizes cross‑stack co‑design as the engine behind modern AI hardware. By formalizing data‑flow languages such as Maestro, his team bridges silicon‑level routing decisions with software‑level abstractions, ensuring flexibility as models shift from CNNs to large language models. Tools like AstroSim, Chakra, and Garnet enable rapid exploration of memory hierarchies, communication fabrics, and accelerator configurations, allowing designers to balance performance with future‑proofing. This layered approach—burning critical data‑flow patterns into silicon while exposing programmable interfaces—helps avoid over‑specialization and supports evolving workloads.

At scale, distributed training introduces new challenges: collective communication, heterogeneous link technologies, and rack‑level fabrics. Krishna discusses how interconnects evolve from on‑chip links to package‑level NVLink‑style connections and finally to rack‑wide networks, each with distinct constraints. Understanding traffic patterns across these layers guides the selection of scalable fabrics that can handle massive model footprints. For enterprises planning AI infrastructure, these insights highlight the importance of holistic design, from custom NPUs to data‑center networking, ensuring that investments remain performant as AI models continue to grow.