OpenAI's latest flagship model, GPT-4o, might actually be regressing, diminishing its performance to that of its smaller variant GPT-40-mini.
According to a new report from Artificial Analysis, OpenAI’s flagship large language model for ChatGPT, GPT-4o, has significantly regressed in recent weeks, putting the state-of-the-art model’s performance on par with the far smaller, and notably less capable, GPT-4o-mini model. This analysis comes less than 24 hours after the company announced an upgrade for the GPT-4o model.
Recommended Videos “We have completed running our independent evals on OpenAI’s GPT-4o release yesterday and are consistently measuring materially lower eval scores than the August release of GPT-4o,” the Artificial Analysis announced via an X post on Thursday, noting that the model’s Artificial Analysis Quality Index decreased from 77 to 71 .
What’s more, GPT-4o’s performance on the GPQA Diamond benchmark decreased from 51% to 39% while its MATH benchmarks decreased from 78% to 69%. Simultaneously, the researchers discovered more than a doubling in the speed increase of the model’s responses, accelerating from around 80 output tokens per second to roughly 180 tokens/s. “We have generally observed significantly faster speeds on launch day for OpenAI models , but previously have not seen a 2x speed difference,” the researchers wrote.
GPT-4o was first released in May 2024 to surpass the existing GPT-3.5 and GPT-4 models. GPT-4o offers state-of-the-art benchmark results in voice, multilingual, and vision tasks, according to OpenAI, making it ideal for advanced applications like real-time translation and conversational AI.
Artificial Intelligence Chatgpt GPT-4O Openai
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
GPT-5: everything we know so far about OpenAI’s next frontier modelGPT-5 is expected to be the next major iteration of OpenAI's large language model. Here's everything we know about it so far.
Read more »
Hospitals use a transcription tool powered by an error-prone OpenAI modelHospitals routinely use a tool powered by OpenAI’s Whisper transcription model, which researchers find can hallucinate entire passages during periods of silence.
Read more »
OpenAI Reported To Launch Its Orion Model In December — Or Maybe NotTor Constantino is a communications professional with 25 years experience as a former journalist and corporate communications executive with an MBA degree. His career has spanned a wide range of technological industries including telecommunications, cloud-based SaaS applications, Big Data analytics, artificial intelligence (AI), and MedTech.
Read more »
OpenAI could release its next-generation model by DecemberA report says OpenAI plans to release its next-generation Orion LLM in December, but CEO Sam Altman is already pushing back.
Read more »
OpenAI plans to release its next big AI model by DecemberOpenAI’s next flagship model, codenamed Orion, is slated to arrive around the two-year anniversary of ChatGPT.
Read more »
Microsoft prepares for OpenAI’s next model as their relationship strainsIf the stars align, we might see OpenAI’s next “Orion” AI model soon.
Read more »