UW CSE (Allen School)

Allen School Colloquium: Productively Programming Accelerated Computing Systems

Rohan, a Stanford PhD and NVIDIA researcher, outlined his work on making high-performance accelerated and distributed computing systems easier to program as hardware grows more heterogeneous and complex. He described a full‑stack approach: high‑level composable distributed libraries that present familiar interfaces (like NumPy) while automatically scaling across clusters; distributed runtime techniques to compose and orchestrate computations efficiently and correctly; and low‑level systems for writing high‑performance kernels across accelerators. He highlighted the Distl compiler for dense and sparse distributed tensor algebra as a concrete example of this strategy. Overall, his research targets both single‑node accelerator specialization and the orchestration challenges of large hierarchical supercomputers.