Steve McDowell is chief analyst at NAND Research. Steve is a technologist with over 25 years of deep industry experience in a variety of strategy, engineering, and strategic marketing roles, all with the unifying theme of delivering innovative technologies into the enterprise infrastructure market.
Cloudian , a long-time leader in scalable object storage technology, directly addresses the complexity of AI storage infrastructure with its new Unveiled at the SC24 supercomputing conference in Atlanta, Cloudian now combines scale and performance in a single solution, simplifying the AI storage landscape with a unified data lake that can act as a shared data repository for all stages of the AI lifecycle.
The new offering is the first object storage solution in the industry to leverage Nvidia GPUDirect technology to reduce latency and improve throughput, meeting the high demands of AI training and other demanding GPU-centric workloads.AI workflows bring unprecedented new data management challenges to IT organizations, as these workflows depend on vast amounts of unstructured data. In addition to the challenges of scale, there is also the complexity of managing a myriad of storage requirements, from training data to checkpoint data, vector databases, log files, and more. Traditional storage solutions, architected for the more modest demands of conventional enterprise applications, struggle to keep up with these demands. Addressing these complexity, cost, and scaling limitations is critical to delivering efficient and cost-effective data management for high-performance AI applications., already the data management foundation for hyperscale cloud providers, offers a clear solution to AI data challenges. Just as it revolutionized cloud infrastructure, object storage transforms AI data management by efficiently handling vast unstructured datasets while maintaining simplicity.: Object storage systems can scale almost infinitely with a flat address space, accommodating multiple data types within a shared data lake.: Leveraging standard server technology and operating at large scales, object storage offers a cost-effective solution for storing and managing the multi-petabyte datasets required for AI model training.Object storage provides comprehensive security for AI workflows through multiple integrated safeguards. Beyond data encryption at rest and in transit, it can offer data immutability for ransomware protection, multi-tenancy, and fine-grained access controls for managing permissions across AI teams.: Native S3 API compatibility and cloud integration simplifies data repatriation from the cloud and ensures interoperability with the many AI tools that employ the S3 API.: Object storage allows for rich, customizable metadata that facilitates efficient data tagging, searching, and management, essential for organizing and retrieving data in AI model training and inference. Cloudian’s on-prem object storage solution, HyperStore, is a scalable, secure, and S3-compatible platform designed to manage vast amounts of unstructured data. HyperStore offers exabyte-level scalability, allowing organizations to start small and expand seamlessly as data requirements grow. Adding GPUDirect technology to the solution lets it better support AI and other high-performance GPU-centric workloads.technology is at the heart of Cloudian’s new HyperStore solution. It enables rapid data transfers between storage and GPU memory by providing athat bypasses traditional CPU routes. This reduces the number of steps and avoids CPU and system memory bottlenecks, thereby accelerating data access.Cloudian’s new HyperStore with Nvidia GPUDirect for Object Storage is a step forward in high-performance data management. The seamless integration of GPUDirect with HyperStore allows users to benefit from Cloudian’s scalability and ease of use while enabling a dedicated high-performance data path for GPU workloads. The Cloudian solution has a compelling performance story. With GPUDirect, Cloudian reports continuous throughput speeds of 200GB/second from a HyperStore system–triple the throughput when using GPUDirect compared to traditional data access methods measured on otherwise identical all-flash systems. This high-performance throughput ensures that data is available to GPUs at the speed required for demanding workloads. By delivering file-system performance directly from object storage, GPUDirect eliminates the need for a separate file-storage layer - simplifying architectures while reducing infrastructure costs. Efficiency improves as well. Cloudian said that it provides an estimated 42% reduction in CPU usage, with GPUDirect freeing up processing power. This makes it easier for organizations to manage large-scale data without increasing their hardware footprint or operational costs, enhancing performance and contributing to lower power consumption and cost savings over time. Cloudian's GPUDirect solution flattens storage architecture into a single data lake, eliminating complex data migrations while simplifying management of vast unstructured datasets. For AI workflows, this means training data, models, and inferencing can all leverage the same high-performance storage platform - accelerating development while ensuring data consistency. There’s also a security story, as GPUDirect technology operates without requiring vendor-specific kernel-level modifications, improving system security by reducing the risk of exposure to vulnerabilities. This is valuable in sectors such as healthcare and finance, where data security and regulatory compliance are paramount.The rapid adoption of AI drives massive data growth, requiring an efficient, scalable, and high-performance storage infrastructure. Cloudian HyperStore with Nvidia GPUDirect for Object Storage addresses this demand by providing an advanced platform optimized for AI and GPU-centric workloads requiring seamless scale data management. Cloudian transforms traditional AI storage architecture by consolidating unstructured data into a single high-performance data lake. The native S3 API and cloud integration facilitate seamless data management between cloud and on-prem environments. Support for Nvidia GPUDirect for Object Storage completes the picture, enabling direct object-to-GPU transfer and eliminating the complexity and cost of separate file and object storage layers. The result is a dramatically simpler infrastructure that delivers better performance and lower operational costs., DDN, WEKA, and VAST Data all support the technology—but Cloudian is the first technology provider to directly integrate GPUDirect into an object storage solution, staying ahead of data trends for AI and other GPU-based workloads. Nvidia GPUs are central to AI-driven enterprise data transformation and are critical components in nearly every phase of the AI lifecycle. Cloudian’s new GPUDirect for object storage solution ensures that enterprises have the tools to harness their data’s full potential. It’s a compelling solution to help enterprises across industries achieve more with their data while controlling costs and enhancing efficiency, a critical capability for enabling AI-driven digital transformation. Disclosure: Steve McDowell is an industry analyst, and NAND Research is an industry analyst firm, that engages in, or has engaged in, research, analysis and advisory services with many technology companies, including every company mentioned in this article, except DDN. No company mentioned was involved in the drafting or publication of this article. Mr. McDowell does not hold any equity position with any company mentioned.Our community is about connecting people through open and thoughtful conversations. We want our readers to share their views and exchange ideas and facts in a safe space.Insults, profanity, incoherent, obscene or inflammatory language or threats of any kindContinuous attempts to re-post comments that have been previously moderated/rejectedAttempts or tactics that put the site security at riskProtect your community.
S3 RDMA Cloudian Dell Technologies DDN VAST Data SC24
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Everything Kalen DeBoer Said As Alabama Moves on to Homecoming WeekThe Alabama head coach meets with the media on Monday after losing to Tennessee.
Read more »
‘Sabrina the Teenage Witch’ Shared a Multiverse With These ABC SitcomsCustom image of Sabrina the Teenage Witch and characters from Boy Meets World by Zanda Rice
Read more »
Emma Corrin Stars in Miu Miu’s 2024 Holiday Campaign “Irreverent Wonder”Where pop culture meets fashion.
Read more »
Pearls, Sequins, & Cigarettes: The World Is Addison Rae’s Oyster & “Aquamarine” Proves ItWhere pop culture meets fashion.
Read more »
‘The Wild Robot’ International Box Office Is About To Pass an Elite MilestoneRoz meets a young Brightbill in &39;The Wild Robot&39;
Read more »
Scarosso Joins Forces With Marta Pozzan for Capsule CollectionWhere pop culture meets fashion.
Read more »