Prevailing AI architectures are not moving the needle. We need new ideas. Google Research proposes NL (nested learning). Here's the AI Insider scoop.
Dr. Lance B. Eliot is a world-renowned AI scientist and consultant.We need to give airtime to new AI architectures if we want to truly advance AI and garner big-time breakthroughs.In today’s column, I examine a fascinating and quite innovative approach to designing and architecting modern AI.
Time will tell whether this represents a substantive and compelling change or whether it might be one of many useful but not definitive side-trips on the pathway to truly advanced AI. The approach is innocuously coined as nested learning . It is perhaps a lot more brash than the name implies. In brief, Google researchers have proposed NL as a means of overcoming the prevailing limitations and constraints of traditional generative AI and large language models . They propose and have built a prototype named Hope that seeks to work on a self-improving basis, employing continual learning, showcasing deeper computational depth, and consisting of interconnected multi-level layers that optimize simultaneously.This analysis of AI breakthroughs is part of my ongoing Forbes column coverage on the latest in AI, including identifying and explaining various impactful AI complexities . It is a large-scale data structure that contains numeric values. The ANN does the bulk of the work when it comes to representing the pattern matching of the written materials that were scanned. An ANN is not the same as a true neural network that exists in your brain, sometimes cheekily referred to as your wetware. The ANN is simplistic and only inspired by some aspects of how the human brain works. I mention this to emphasize that though many in the media tend to equate ANNs with real NNs, it is not a fair comparison. For more details on ANNs and how they function, see my discussion atBy and large, once an AI developer has done the initial setup of the LLM, it will remain relatively the same until the AI developer comes along to make further changes to it. Most of the common LLMs do not self-adjust in real-time. They are instead adjusted by AI developers, from time to time, and otherwise are relatively static.When you use a popular LLM such as ChatGPT, GPT-5, Claude, Gemini, Grok, etc., the AI is pretty much basing what it figures out via the initial data training that originally took place. That’s the main corpus of the pattern matching. Pretend for a moment that the only scanned content for a particular LLM on the topic of baseball consisted of the rules of the sport. Just the barebones rules. There wasn’t anything available to be scanned about the advanced aspects of baseball. Nor was there any data scanned about coaching baseball players and teams. And so on.Please know that I hesitate to ask whether the LLM can “learn” more about baseball, and instead phrased this updating action as an adjustment or improvement. I do so to try and avoid anthropomorphizing AI. Allow me to elaborate. The word “learn” is usually associated with humans and what humans do in their heads. AI is not doing this the same way that we do in our minds. In that manner, it is a bit misleading to refer to AI as “learning” – but everyone uses that phrasing anyway since it is convenient. I will reluctantly proceed to use the word “learn” with respect to AI, but now you know that I mean the word as it relates to AI mathematically and computationally, and not to be equated with the magic that occurs inside the human noggin.If you were to enter prompts into our pretend LLM and ask about the fundamentals of baseball, you would probably be satisfied with the response. That is what was contained in the initial setup. But if you ask advanced questions about baseball, the AI will tell you that there isn’t anything else about baseball that it can say. You would almost certainly stymy the AI if you asked how to coach baseball players and baseball teams. This is because there isn’t anything there for the AI to retrieve or rely upon. You can temporarily overcome this paucity by entering prompts to tell the AI more about the topic of baseball. If the AI is connected to the Internet for web searching, it could also go look up more data about baseball. Another means of infusing data would be to use in-context modeling or RAG , which allows you to import documents into the AI as additional data sources. See my explanation about in-context modeling and RAG atThe thing is that those materials are usually only temporarily utilized by the AI. The LLM isn’t going to on-the-spot permanently “learn” from those inputted aspects of baseball. It will seem to have ingested the data during your conversations, but this isn’t being incorporated on a permanent basis into the totality of the AI system.If a friend of yours logs into the AI and asks about baseball, the only aspects they will see will be the fundamentals that were gleaned during the initial overall setup. Your conversations about baseball have not automatically caused the AI to update across the board. We might say that the AI hasn’t been able to learn from your conversations and inputs about baseball. That’s a bummer. It sure would be nifty if the AI could automatically learn and adjust based on the millions upon millions of people interacting with the AI. Imagine the incredible possibilities! Downsides exist. Suppose the AI learns falsehoods. This could easily happen. Someone tells the AI that in baseball, a player can skip third base and run directly to home plate . The AI might be fooled or tricked. Meanwhile, if this is infused in the totality of the AI, the AI will repeat that falsehood to millions of other users. Not good. Learning is a dicey proposition. That’s why the norm consists of AI developers opting to adjust and improve the AI, refreshing it and updating it, therefore doing the act of learning for the AI by guiding the AI in doing so.A cogent argument can be made that contemporary AI is not going to attain artificial general intelligence unless we find a suitable means for AI to undertake self-learning : “Despite all their success and remarkable capabilities in diverse sets of tasks, LLMs are largely static after their initial deployment phase, meaning that they successfully perform tasks learned during pre- or post-training, but are unable to continually acquire new capabilities beyond their immediate context.” The only adaptable component of LLMs is their in-context learning ability -- a characteristic of LLMs that enables fast adaptation to the context and so perform zero- or few-shot tasks.” “In this paper, we present a new learning paradigm, called Nested Learning , that coherently represents a model with a set of nested, multi-level, and/or parallel optimization problems, each of which with its own ‘context flow’.” “NL reveals that existing deep learning methods learns from data through compressing their own context flow and explain how in-context learning emerges in large models.” “NL suggests a path to design more expressive learning algorithms with more ‘levels’, resulting in higher-order in-context learning abilities.”For those of you versed in the technical underpinnings of AI, I suggest you consider reading the research paper to get the eye-popping details. Their viewpoint is that NL provides a new dimension to the design of AI models. For example, they model backpropagation as a form of associative memory. Likewise, transformer attention mechanisms are devised as associative memory modules. They use a defined frequency rate for when to update weights, serving as a means to arrange the interconnected optimizations into various levels. Another novelty is an extension of feedforward ANNs into a paradigm they coin as a CMS . In turn, this form of a memory system of a long-term nature is crucial to enabling continual learning. They have constructed a proof-of-concept named Hope that can be used in experiments to gauge how well this works and can spur additional enhancements by interested AI developers.I’ve repeatedly noted in my column and in my many presentations that we are boxed in when it comes to prevailing AI architectures. Though some believe that we only need to toss more and faster hardware at the existing AI to get it to reach the heights of AGI, I seriously doubt this. That’s why I embrace out-of-the-box attempts to legitimately discover alternative architectures, see for example my coverage at
Generative AI Large Language Model LLM Openai Chatgpt GPT-5 GPT-4O Anthropic Claude Google Gemini Meta Llama Xai Grok Architecture Design Build Construct Test Field Neural Network NN ANN Associative Memory Long-Term Short-Term Layers Advances Breakthroughs New Ideas
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Inside the Visual Strategy That Won NYC For Zohran MamdaniElectric blues and fiery oranges made the campaign impossible to ignore across the city.
Read more »
'GMA' gets a sneak peek inside Disney Cruise Lines' new ship, the Disney DestinySee the heroes and villains theme of Disney's brand-new cruise ship, the Destiny.
Read more »
11 Freaky Sci-Fi Hive Minds From Movies and TVApple TV's excellent new series 'Pluribus' explores a seemingly harmonious, deeply harrowing take on the classic trope.
Read more »
SNAP benefits battle is weighing heavily on the minds of NYC homebound seniorsEven though the longest government shutdown in history may be nearing an end, there is still a lot of uncertainty for food stamp recipients.
Read more »
70-Million-Year-Old Dinosaur Egg Reveals Ancient Nesting CluesDiscover Magazine’s award-winning journalism inspires and informs, delivering thought-provoking content that sparks meaningful conversations. With a focus on groundbreaking developments in science, technology, and the world around us, Discover highlights the impact of these innovations on our daily lives.
Read more »
The Psychiatrist Who Studied Nazi Officials After the War Found Something Shocking. It Didn’t Earn Him Any Fans.Understanding the minds of monsters is no easy task.
Read more »