Epoch AI

Total AI Chip Memory Bandwidth Has Grown 4.1x per Year, Now Reaching 70 Million TB/S

AI chip memory bandwidth has accelerated to 70 million terabytes per second, representing a 4.1× annual growth since 2022. This capacity dwarfs global internet traffic by a factor of roughly 300,000, highlighting the massive data movement required for modern inference. The surge has strained high‑bandwidth memory (HBM) supply, pushing prices higher in early 2026, with AI chips consuming over 90 % of HBM output in 2025. Tracking bandwidth offers a clearer view of the world’s ability to serve increasingly large AI models.