This article was written by Cambrian-AI Analyst Alberto Romero and Karl D-Matrix was founded in 2019 by two veterans in the field of AI hardware, Sid Sheth and Sudeep Bhoja, who previously worked together at Inphi (Marvell) and Broadcom.
Startup combines Digital In-Memory Compute and chiplet implementations for data-center-grade inferencing.D-Matrix was founded in 2019 by two veterans in the field of AI hardware, Sid Sheth and Sudeep Bhoja, who previously worked together at Inphi and Broadcom.
The company was born at a singular moment for the field of AI, just two years after the popular transformer architecture was invented by Google Brain scientists. By 2019, the world was starting to realize the massive significance of transformer-based models and D-Matrix saw an opportunity to define its AI hardware specifically to excel in using these Large Language Models.GPT-3, MT-NLG, Gopher, DALL·E, PaLM, and virtually every other large language model is based on the now ubiquitous transformer architecture. Tech companies keep announcing potentially amazing models that remain inaccessible to the world due to one insurmountable obstacle: deploying these models into production for inference at the data center is virtually unfeasible with current AI hardware. That’s what D-Matrix is aiming to solve and, as a company developing in parallel to the already world-changing wave of transformers and LLMs, they’re well-posited to bring a clean-slate approach to this problem. Focusing on large multimodal models is what differentiates the company from its competitors. Transformer-based models are usually trained on high-performance GPUs , but performing inferences is a power efficiency story, not just performance at any cost. D-Matrix has found an innovative solution with which they claim to achieve 10–30x the efficiency of the current hardware. Once tech companies begin to embed transformer-based NLP models in all kinds of applications and spread them across industries, this type of ultra-efficient hardware will be appealing to handle the inference workloads.D-Matrix’s solution is currently a proof-of-concept chiplet-based architecture called Nighthawk. Together with Jayhawk, its soon-to-be second chiplet that will also implement die-to-die interfaces, they form the basis for Corsair, D-Matrix’s hardware product planned to be released in the second half of 2023. Nighthawk comprises an AI engine with four neural cores and a RISC-V CPU. Each neural core is composed of two octal compute cores , each of which has eight digital in-memory compute cores where weights are stored, and matrix multiplication is performed. Nighthawk emerges from the novel combination of three technological pillars. First is digital in-memory compute . The efficiency barrier that existing hardware suffers is due to the costs and performance limits caused by moving data around to do the computations. D-Matrix has mixed the accuracy and predictability of digital hardware with super-efficient IMC to create what D-Matrix believes is the first DIMC architecture for inference at the data center. Nighthawk’s projected performance seems to back D-Matrix’s idea of bringing both data and compute into the SRAM, which is the current best memory type that serves the IMC solution. D-Matrix claims its hardware is 10x more efficient than an NVIDIA A100 for inference workloads.
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
AI: The Somnium Files - nirvanA Initiative Review - Mad, Criminal BrillianceAI: The Somnium Files - nirvanA Initiative is a brilliant, captivating, and entertaining murder mystery that's a must-play. Our review of the latest from SpikeChunsoft_e:
Read more »
AI-Driven Fitness: Making Gyms Obsolete?Surprisingly, fitness just might be easier to achieve during a global pandemic.
Read more »
Coatue Adds Atlassian CTO As New Partner Investing In AICareer operator Sri Viswanath previously worked at Groupon and VMware.
Read more »
Brightdrop Buys AI Company To Help Customers Make The Switch To ElectricGM's Brightdrop division, a unit focused on electric work vans, recently bought a company that could give it an edge in artificial intelligence and other software capabilities. “We’re bringing entirely new ways of doing business
Read more »
AI Ethics Leans Into Aristotle To Examine Whether Humans Might Opt To Enslave AI Amidst The Advent Of Fully Autonomous SystemsThe inaugural annual presentation of the Oxford University Institute for Ethics and AI featured a Stanford talk covering Aristotle and lessons for AI Ethics, especially dealing with the controversial idea that perhaps humans will opt to enslave sentient AI, if we ever get there.
Read more »
Microsoft will phase out facial recognition AI that could detect emotions | EngadgetMicrosoft is shelving facial recognition AI it says could detect your emotions and age..
Read more »