Nvidia's generative AI inferencing card is just two H100s glued together
That performance can likely be attributed to Nvidia's decision to use faster HBM3 memory instead of HBM2e. We'll note that Nvidia is already using HBM3 on its larger SMX5 GPUs. And the memory doesn't just have higher bandwidth — 4x compared to a single 80GB H100 PCIe card — there's also more of it: 94GB per die.
While Nvidia already has its larger SXM5 H100s in the wild for AI training, those are only available from OEMs in sets of four or eight. And at 700W a piece, those systems are not only hot, but potentially challenging for existing datacenters to accommodate. For reference, most colocation racks come in at betweenBy comparison, the H100 NVL, at 700W, should be a bit easier to accommodate. By our estimate a single socket, dual H100 NVL system would be somewhere in the neighborhood of 2.5KW.
"It's for efficient AI, video, and graphics," Buck said, adding that the card is specifically optimized for AI video workloads and features new encoder/decoder accelerators. This functionality lines up with existing 4-series cards from Nvidia, which have traditionally been used for video decoding, encoding, transcoding, and video streaming.
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Watch Nvidia's GTC 2023 keynote right here todayIt's going to be AI as far as the digital eye can see at Nvidia's GTC keynote today. Kicking off in 10 minutes time!
Read more »
Oops! Leaked RTX 4070 and 4060 GPUs spoil Nvidia’s partyRTX 4070 and 4060 accidently revealed – and Nvidia must be fuming
Read more »
Unpopular Windows 11 Start menu change could be back in the cardsCould a 'recommended websites' feature still come to Windows 11?
Read more »
Mitrovic rages at referee as Fulham collapse with three red cards at Man UnitedFulham had Aleksandar Mitrovic, Willian and manager Marco Silva sent off in the space of a minute as they imploded at Manchester United. The Cottagers had been on course for a famous FA Cup win wit…
Read more »
Celebrity Mother's Day celebrations from throwback snaps to handmade cardsAs Sunday March 19th once again marks the return of Mothering Sunday, some of our favourite stars have marked the special day with a slew of tributes, photos and heartwarming gifts
Read more »
Prince William children's moving Mother's Day tributes to DianaTouching messages written by George, Charlotte and Louis to their late grandmother Diana in old Mother's Day cards have resurfaced
Read more »