Technique could be used as a cheap way to boost ability of other AI models. Technique could be used as a cheap way to boost ability of other AI models.
Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser . In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.
A Tiny Reasoning Model beat Large Language Models in solving logic puzzles, despite being trained on a much smaller dataset.A small-scale artificial-intelligence model that learns from only a limited pool of data is exciting researchers for its potential to boost reasoning abilities. The model, known as Tiny Recursive Model , outperformed some of the world’s best large language models at the— is not readily comparable to an LLM. It is highly specialized, excelling only on the type of logic puzzles on which it is trained, such as sudokus and mazes, and it doesn’t ‘understand’ or generate language. But its ability to perform so well on so few resources — it is 10,000 times smaller than“It’s fascinating research into other forms of reasoning that one day might get used in LLMs,” says Cong Lu, a machine-learning researcher formerly at the University of British Columbia in Vancouver, Canada. However, he cautions that the techniques might no longer be as effective if applied on a much larger scale. “Often techniques work very well at small model sizes and then just stop working,” at a bigger scale, he says.“The results are very significant in my opinion,” says François Chollet, co-founder of AI firm Ndea, who created the ARC-AGI test. Because such models need to be trained from scratch on each new problem, they are “relatively impractical”, but “I expect a lot more research to come out that will build on top of these results”, he adds. The sole author of the paper — Alexia Jolicoeur-Martineau, an AI researcher at Samsung AI Lab in Montreal, Canada — says that her model shows that the idea that only massive models that cost millions of dollars to train can succeed at hard tasks “is a trap”. She has made the model’s codefor anyone to download and modify. “Currently, there is too much focus on exploiting LLMs rather than devising and expanding new lines of direction,” she wroteare built on top of LLMs, which predict the next word in a sequence by tapping into billions of learned internal connections, known as parameters. They excel by memorizing patterns from billions of documents, which can trip them up when they come to unpredictable logic puzzles. The TRM takes a different approach. Jolicoeur-Martineau was inspired by a technique known as the hierarchical reasoning model, developed by the AI firm Sapient Intelligence in Singapore. The hierarchical reasoning model improves its answer through multiple iterations and was published in a preprint in June The TRM uses a similar approach, but uses just 7 million parameters, compared with 27 million for the hierarchical model and billions or trillions for LLMs. For each puzzle type the algorithm learns, such as a sudoku, Jolicoeur-Martineau trained a brain-inspired architecture known as a neural network on around 1,000 examples, formatted as a string of numbers.During training, the model guesses the solution and then compares it with the correct answer, before refining its guess and repeating the process. In this way, it learns strategies to improve its guesses. The model then takes a similar approach to solve unseen puzzles of the same type, successively refining its answer up to 16 times before generating a response.Access the most recent journalism from Nature's award-winning teamFaculty Positions in AI for Life Sciences at Westlake University Invites applications for tenure-track or tenured faculty positions at all academic ranks in the field of artificial intelligence for life sciences.Job Description Job Title: Associate or Senior Editor, Nature Chemical Biology Organization: Nature Portfolio Location: New York, Washington DC, Je...
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Advancing AI By Nesting Minds Inside The Layers Of Machine Learning And LLMsPrevailing AI architectures are not moving the needle. We need new ideas. Google Research proposes NL (nested learning). Here's the AI Insider scoop.
Read more »
DOG WALKING FORECAST: Tiny jumps in temperatures todayMeteorologist at News 5 Cleveland
Read more »
95% success rate: Tiny robots deliver drugs directly to clots to fight strokes, tumorsThe ETH Zurich team has developed magnetic microrobots capable of delivering medication directly to specific targets within the body.
Read more »
These tiny robots swim through your blood to fight strokesAndrew Paul is Popular Science's staff writer focused primarily on tech, AI, physics, and culture news. He was previously a regular contributor to The A.V. Club and Input, and has been featured by Rolling Stone, Fangoria, GQ, Slate, NBC, McSweeney's Internet Tendency, and elsewhere. He lives outside Indianapolis.
Read more »
Queens self-defense class for Muslim women teaches more than just techniqueAt the Malikah Safety Center in Astoria, Queens, Muslim women are learning more than just self-defense.
Read more »
Strain technique unlocks triple-phase behavior to advance safer lead-free ferroelectricsScientists use strain to boost lead-free ferroelectrics, creating triple phases that enhance performance for safer devices.
Read more »