USENIX Association

NSDI '26 - Co-Designing Traffic Control with NVMe-oF for Disaggregated Storage: A Comparative Study

The presentation compares two disaggregated‑storage fabrics— a traditional switched SAN and a novel switchless mesh— and proposes traffic‑control mechanisms co‑designed with NVMe‑of. By decoupling compute and storage, modern data centers face rapidly scaling PCIe bandwidth and higher drive density, prompting a choice between expensive high‑port switches and cheaper multi‑hop adapters. Three observations drive the design: network round‑trip time is a tiny fraction of total I/O latency, NVMe‑of traffic is inherently pairwise and asymmetric, and SSD performance throttles the entire fabric, causing back‑pressure that fills switch buffers. Existing congestion‑control schemes ignore these traits, so the authors introduce in‑network telemetry‑assisted symmetric routing, eager bandwidth reservation, and storage‑driven traffic scheduling. Experiments on four nodes equipped with Samsung PM93 SSDs show that both architectures deliver similar throughput, but the switchless mesh reduces average latency by 20‑28% and tail latency by about 25% across YCSB workloads. Cost analysis reveals that at 1,000 ports the switchless design saves roughly 50% of capital expenditure compared to a fully switched fabric. The findings suggest that, for disaggregated storage, a switchless topology combined with traffic‑aware control can deliver lower latency and substantial cost savings without sacrificing performance, offering a practical blueprint for next‑generation data‑center storage networks.