Recent research out of UC Santa Cruz shows that modern LLMs running billions of parameters can operate on just 13 watts of power without a loss in performance.
The large language models that power today’s chatbots like ChatGPT, Gemini, and Claude are immensely powerful generative AI systems, and immensely power-hungry ones to boot.
Recommended Videos “We got the same performance at way less cost — all we had to do was fundamentally change how neural networks work,” lead author of the paper, Jason Eshraghian, said. “Then we took it a step further and built custom hardware.” They did so by doing away with the neural network’s multiplication matrix.
These matrices are stored on hundreds of physically separate GPUs and fetched with each new query or operation. The process of shuttling data that needs to be multiplied among the multitude of matrices costs a significant amount of electrical power, and therefore money. “From a circuit designer standpoint, you don’t need the overhead of multiplication, which carries a whole heap of cost,” Eshraghian said. And while the team did implement its new network on custom FGPA hardware, they remain confident that many of the efficiency improvements can be retrofitted to existing models using open-source software and minor hardware tweaks. Even on standard GPUs, the team saw a 10 times reduction in memory consumption while improving operational speed by 25%.
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
'Cape of Storms': Climate researchers explain Cape Town's recent extreme weatherA severe storm hit South Africa's Western Cape province between 6 and 9 April 2024, with extreme winds gusting at up to 135km/h. The storm left a trail of destruction across Cape Town and surrounding areas—at least 1,500 people were left homeless after the high winds fanned fires through their communities, burning homes to the ground.
Read more »
New method uses light to enable the generation of non-canonical amino acidsUC Santa Barbara researchers are building out the repertoire of chemical reactions, using light.
Read more »
Money from Ted Cruz podcast continues to flow to pro-Cruz PACThe $156,186 in “digital revenue” was sent in May from the company that distributes Cruz’s podcast to Truth and Courage PAC, which is working toward Cruz's...
Read more »
Santa Cruz jailhouse shanking defendant faces 7 years in plea dealThe second of three co-defendants in an attempted jailhouse murder case has pleaded no contest to charges ahead of his preliminary hearing.
Read more »
Santa Cruz Beach Boardwalk Roller Coaster Evacuated, Riders Have to Walk DownThrill seekers got more than they bargained for at the Santa Cruz Beach Boardwalk ... folks who were riding the classic roller coaster had to be evacuated near the top of the ride!!!
Read more »
Santa Cruz County man rescued from remote canyon after nine-day search“About 10 days he survived in the wilderness, essentially, drinking out of the creek and eating wild berries,” Boulder Creek Fire Chief Mark Bingham said. “He did have a minor inj…
Read more »