Tech Product Reviews, How To, Best Ofs, deals and Advice
It has been nearly two years since Microsoft CEO Satya Nadella predicted that generative AI would take over knowledge work, but if you look around a typical law firm or investment bank today, the human workforce is still very much in charge.
Despite all the hype about “reasoning” and “planning,” a new study from training-data company Mercor explains exactly why the robot revolution is stalled: AI just can’t handle the messiness of real work. A reality check for the “replacement” theory Mercor released a new benchmark called APEX-Agents, and it is brutal. unlike the usual tests that ask AI to write a poem or solve a math problem, this one uses actual queries from lawyers, consultants, and bankers. It asks the models to do complete, multi-step tasks that require jumping between different types of information. Recommended Videos The results? Even the absolute best models on the market—we are talking about Gemini 3 Flash and GPT-5.2—couldn’t crack a 25% accuracy rate. Gemini led the pack at 24%, with GPT-5.2 right behind it at 23%. Most others were stuck in the teens. Why AI is failing the “office test” Mercor CEO Brendan Foody points out that the issue isn’t raw intelligence; it’s context. In the real world, answers aren’t served up on a silver platter. A lawyer has to check a Slack thread, read a PDF policy, look at a spreadsheet, and then synthesize all that to answer a question about GDPR compliance. Humans do this context-switching naturally. AI, it turns out, is terrible at it. When you force these models to hunt for information across “scattered” sources, they either get confused, give the wrong answer, or just give up entirely. The “Unreliable Intern” For anyone worried about their job security, this is a bit of a relief. The study suggests that right now, AI functions less like a seasoned professional and more like an unreliable intern who gets things right about a quarter of the time. That said, the progress is terrifyingly fast. Foody noted that just a year ago, these models were scoring between 5% and 10%. Now they are hitting 24%. So, while they aren’t ready to take the wheel yet, they are learning to drive much faster than we expected. For now, though, the “knowledge work” revolution is on hold until the bots learn how to multitask p
Artificial Intelligence Gemini Microsoft Openai Satya Nadella
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Your next road trip is booked: Forza Horizon 6 comes this MayTech Product Reviews, How To, Best Ofs, deals and Advice
Read more »
Here’s when you can buy AMD’s Ryzen 7 9850X3D and how much it’ll costTech Product Reviews, How To, Best Ofs, deals and Advice
Read more »
Key moment approaches for NASA’s crewed moon missionTech Product Reviews, How To, Best Ofs, deals and Advice
Read more »
You might actually be able to buy a Tesla robot in 2027Tech Product Reviews, How To, Best Ofs, deals and Advice
Read more »
1Password helps prevent your passwords from going to scam sitesTech Product Reviews, How To, Best Ofs, deals and Advice
Read more »
Alder Lake is ending and here’s what it means for your current PCTech Product Reviews, How To, Best Ofs, deals and Advice
Read more »