Nvidia’s Dynamo enables faster and more efficient AI inference with scalable digital storage and memory hierarchical KV caching. More to come.
At the 2025 Nvidia GPU Technology Conference the company announced its AI Data Platform that included significant advances in enterprise digital storage to support corporate AI workloads. However, the company’s KV cache in its Dynamo software and future looking efforts to connect storage and memory more directly with GPUs will drive digital storage and memory demand further, improve inference performance and lower AI costs.
The AI Data Platform leverages Blackwell GPUs, BlueField DPUs and Spectrum-X networking to deliver 1.6X higher performance than CPU-based storage, reducing power consumption by up to 50% and providing more than three types higher performance per watt and accelerating storage traffic up to 48% compared to traditional Ethernet. This roll out was done in conjunction withTom Coughlin In a recent conversation with Kevin Deierling from Nvidia we discussed another topic related to storage announcements at the 2025 GTC, that is the Key Value Cache in Nvidia’s Dynamo, see image below of Jensen announcing Dynamo.Today’s NYT Mini Crossword Clues And Answers For Wednesday, May 7th Jensen characterized Dynamo as the OS of the AI factory. These key values are binary representations of the state of the AI model at a point in time. This KV cache grows to become very large for large models. But the KV cache allows faster user responses and avoids the need to recalculate model results and thus reduces costs and increases efficiency. Nvidia Dynamo is open-source high-throughput low-latency inference software that is intended to standardize model deployment and enables fast and scalable AI in production. Because creating trained KV values for user requests is compute intensive and keeping them solely on GPU memory is expensive the Dynamo KV Cache Manager enables the offloading of older or less frequently access KV cache blocks to more cost-effective memory and storage such as CPU memory, local storage or networked object or file storage. This enables organizations to cost-effectively store petabytes of KV cache data by distributing KV cache blocks between a hierarchy of GPU accessed storage as shown below, depending upon frequency of use. Such a hierarchy will include memory as well as SSDs and HDDs. Dynamo can manage KV cache across multiple GPU nodes and supports both distributed and disaggregated inference serving with the hierarchical caching creating offloading strategies at multiple levels.There is another effort that Nvidia and several digital storage and memory companies are working on that has been called Storage Next. This is an initiative within the Open Compute Project to create a new storage architecture for GPU computing hear memory for disaggregated data-protected, managed block storage using next generation NVMe over the PCIe generation 6 bus. This is expected to provide lower total cost of ownership, higher IOPS, lower power consumption and less complex infrastructure and reduced impact from tail latencies. Kevin’s comment to me was that this will include computational storage for AI. Nvidia plans to talk further about this effort at the 2025 FMS in August. Nvidia’s Dynamo enables faster and more efficient AI inference with scalable digital storage and memory hierarchical KV caching. Work in development by the storage industry will allow even tighter integration of digital storage with GPUs.
Dynamo AI Data Platform KV Cache Bluefield Blackwell Digital Storage Hdd Ssd Jensen
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Diego Fagúndez, short-handed Galaxy play Dynamo to 1-1 drawDiego Fagúndez scored a goal for LA and the short-handed Galaxy tied the Houston Dynamo 1-1.
Read more »
Ondřej Lingr’s stoppage-time goal in MLS debut helps Dynamo tie Rapids 2-2Matt Schubert is the Sports Editor for The Denver Post. He is a graduate of the Cronkite School of Journalism and Mass Communication at Arizona State University. His journalism career has spanned three time zones and four states, with previous stops in Washington, Nebraska and Indiana.
Read more »
Nathan Ordaz, Jeremy Ebobisse lead LAFC past DynamoOrdaz scores early and Ebobisse adds insurance in the second half as LAFC stretches its unbeaten run to four matches with a 2-0 win over Houston.
Read more »
Nvidia and Anthropic Clash over AI Chip Exports to ChinaSource of breaking news and analysis, insightful commentary and original reporting, curated and written specifically for the new generation of independent and conservative thinkers.
Read more »
Woman escapes storage container she was being held in after brutal attack, deputies sayA man in Florida was arrested after a woman he allegedly assaulted and held in a storage container escaped.
Read more »
The Storage Scam: How Apple, Google and Samsung overcharge you for storageVictor, a seasoned mobile technology expert, has spent over a decade at PhoneArena, exploring the depths of mobile photography and reviewing hundreds of smartphones across Android and iOS ecosystems.
Read more »