'We are not here to jerk ourselves off about parameter count.'
Generally speaking, when it comes to AIs and LLMs in particular, bigger has been better. OpenAI's first landmark model GPT-2, released in 2019, boasted around 1.5 billion parameters, the adjustable variables that connect the neurons of an AI that help it "learn" and refine itself based on its input data.
By the time GPT-3 rolled out the next year, it boasted a whopping 175 billion parameters, and by GPT-4, one trillion, according to somenotes, OpenAI itself has not shared GPT-4's exact size, which is perhaps emblematic of the company's pivot away from simply enlarging its models. While each of these increases in parameters has been met with an uptick in the capabilities of the GPT models, this approach may now be yielding diminishing returns, according to theStill Heading Up
It's worth mentioning that Altman conceded that parameter counts may trend up regardless — diminishing returns, after all, are still returns — but he maintains that the metric gets "way too much focus." "This reminds me a lot of the gigahertz race in chips in the 1990s and 2000s, where everybody was trying to point to a big number," Altman said,"What we want to deliver to the world is the most capable and useful and safe models," he added. "We are not here to jerk ourselves off about parameter count."
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
OpenAI’s CEO Says the Age of Giant AI Models Is Already OverSam Altman says the research strategy that birthed ChatGPT is played out and future strides in artificial intelligence will require new ideas.
Read more »
Elon Musk says he wants to start ‘TruthGPT’ to rival OpenAI and Google | EngadgetElon Musk’s plans for his own AI company are starting to come into focus..
Read more »
This CEO Has A Better Rule For Making Faster DecisionsLeaders have to make decisions; not everything can be delegated. But by focusing on a very specific set of decisions, this CEO has cracked the code on making faster and better decisions.
Read more »
The Very Best Lines From Season 4, Episode 4 of ‘Succession’Gerri for CEO. Gerri for CEO of everything!
Read more »
Amazon CEO Andy Jassy is no longer an 'overpaid CEO': advocacy groupAmazon CEO Andy Jassy is no longer an 'overpaid CEO' as his pay dipped in 2022, shareholder advocacy group says
Read more »
Microsoft's $13 billion bet on OpenAI carries huge potential along with plenty of uncertaintyMicrosoft's partnership with OpenAI could mean billions of dollars a year in new revenue, but the investment structure is complicated.
Read more »