OpenAI, Microsoft And Friends Build A Better, More Scalable Ethernet

Why It Matters

MRC reshapes AI‑cluster networking by delivering scalable, low‑latency performance at lower cost, reducing downtime risk and enabling larger supercomputers without the expense of next‑gen high‑bandwidth switches.

Key Takeaways

• MRC replaces high‑bandwidth ports with many low‑speed links, cutting latency
• Two‑tier MRC topology halves switch hops, doubling compute nodes per switch budget
• Adaptive spraying and trimming keep AI training running despite link failures
• Implemented on Nvidia, AMD, Broadcom SmartNICs and major switch operating systems
• Early deployments at Oracle Stargate and Azure AI datacenters show near‑zero congestion

Pulse Analysis

The emergence of Multipath Reliable Connection (MRC) marks a strategic shift in data‑center networking for artificial‑intelligence workloads. Rather than chasing ever‑higher port speeds, MRC leverages higher‑radix switch ASICs to spread traffic across eight low‑speed links per endpoint. This architecture reduces the number of required switch tiers, slashing hop counts from five‑seven to three and delivering lower latency while keeping total bandwidth per GPU or XPU unchanged. For hyperscalers, the cost savings are two‑fold: fewer high‑end switches to purchase and lower power consumption, directly improving total cost of ownership.

MRC’s protocol enhancements—adaptive packet spraying, explicit congestion notification, and packet‑trimming—address long‑standing congestion and reliability challenges in RoCE‑based Ethernet. By dynamically redistributing traffic across multiple links and retransmitting only the missing payload, the network can absorb link failures without halting training jobs. This resilience is critical when scaling to hundreds of thousands of GPUs, where a single failed link in traditional Clos topologies could force a costly checkpoint restart. The static IPv6 segment routing further simplifies path management, allowing eight parallel data planes to operate without complex dynamic routing.

Industry adoption is already underway. Nvidia’s ConnectX‑8 SmartNICs, AMD’s Pollara and Vulcano DPUs, and Broadcom’s Thor Ultra SmartNICs have integrated MRC, while major switch vendors such as Nvidia (Spectrum 4/5), Arista (EOS) and others support the required SRv6 static routing. Deployments at OpenAI’s Oracle Stargate facility in Texas and Microsoft’s Azure AI datacenter in Wisconsin demonstrate tangible performance gains—near‑zero congestion and uninterrupted training despite link outages. As AI models grow in size and training cycles lengthen, MRC offers a pragmatic, cost‑effective path to scale beyond the limits of traditional high‑bandwidth Ethernet, positioning participating vendors as key enablers of the next generation of AI supercomputers.