Beyond the GPU: Cisco Says AI’s Biggest Challenge May Be the Network That Connects It All

Why It Matters

Network capacity is becoming a differentiator for AI performance, affecting latency, cost and the ability of hyperscalers and service providers to deliver next‑generation AI services at scale.

Key Takeaways

• AI traffic uses ~30% of backbone, up from <1% two years ago
• Cisco’s Agile Services Networking merges Layer 1‑3 functions into coherent optics
• Unified Edge delivers GPUs, storage, networking in 3‑5RU form factor
• Providers jumping from 10 Gb to 400 Gb still fall short of AI demand

Pulse Analysis

The AI boom has long been measured in GPU cores, megawatts and cooling capacity, but network bandwidth is emerging as the next bottleneck. Data from Cisco’s Robin Olds shows AI workloads now occupy roughly 30 % of backbone utilization—a stark rise from under 1 % just two years earlier. This shift is driven not only by massive training clusters but also by the rise of autonomous AI agents that maintain a constant stream of inference requests. As baseline traffic climbs, operators must redesign capacity planning to accommodate sustained, rather than peak‑only, demand.

Cisco is answering the challenge with its Agile Services Networking framework, which collapses traditional Layer 1, Layer 2 and Layer 3 functions into a single coherent‑optics platform. By embedding routing intelligence directly into the optical layer, the solution trims equipment footprints, cuts power draw and frees rack space for additional compute. The company’s Unified Edge portfolio extends this philosophy to the edge, packaging NVIDIA, AMD or Intel accelerators, storage and networking into compact 3‑RU or 5‑RU chassis. Such integrated nodes turn central offices and metro sites into mini‑data centers capable of on‑site inference.

The implications reach beyond individual operators. A recent case where a provider leapt from 10 Gb to 400 Gb links—skipping the 100 Gb tier—still found capacity insufficient as AI demand tripled the newly provisioned bandwidth. This illustrates that AI‑driven traffic is outpacing traditional upgrade cycles, making the middle mile a strategic battleground. Coordinated investments across hyperscalers, neoclouds and service providers will be essential to build a low‑latency, high‑throughput fabric that can sustain the next generation of AI services.