Know What's Happening in Hardware

SK Hynix Proposes HBM and HBF Hybrid for LLM Inference

SK Hynix unveiled H³, a hybrid architecture that couples high‑bandwidth memory (HBM) with high‑bandwidth flash (HBF) on a single interposer to accelerate large‑language‑model (LLM) inference. HBF provides up to 16× the capacity of HBM while maintaining comparable bandwidth, serving as a fast‑access cache for read‑only KV data. Simulations with an Nvidia Blackwell GPU showed token‑throughput gains of 1.25× to 6.14×, a 2.69× improvement in performance‑per‑watt, and an 18.8× increase in batch size versus HBM‑only systems. The design targets read‑only KV‑cache workloads, reducing GPU count and power consumption.