Nicole Hemsoth

Dynamo + VAST Makes KV Cache Shareable, Saves GPU Memory

Inference doesn’t break on compute these days, it's GPU memory that's the choke. But Dynamo + VAST turns KV cache into a first-class, shareable system resource so GPUs stop redoing context. Cool stuff https://t.co/4v0Au4rcM5