DDN touts Infinia storage as key to faster, cheaper AI inference
Blocks & Files, Saturday, July 19th, 2025
DDN has released performance benchmarks showing it can can speed up AI processing time by 27x because of the way it handles intermediate KV caching.
An AI LLM or agent, when being trained on GPUs or doing inference work on GPUs and possibly CPUs, stores existing and freshly computed vectors as key-value items in a memory cache, the KV cache. This can have two memory tiers in a GPU server; the GPUs' HBM and the CPUs' DRAM. If more data enters the KVCache, existing data is evicted.
If needed later, it has to be recomputed or, if moved out to external storage, such as locally attached SSDs or network-attached storage, retrieved, which can be faster than recomputing the vector. Avoiding KV cache eviction and recomputation of vectors is becoming table stakes for AI training storage vendors, with DDN, Hammerspace, VAST, and WEKA as examples.