Dr. Lance B. Eliot is a world-renowned expert on artificial intelligence with over 8.1+ million amassed views of his AI columns and been featured on CBS 60 Minutes. As a CIO/CTO seasoned executive and high-tech entrepreneur, he combines practical industry experience with deep academic research.
In today’s column, I closely explore the rapidly emerging advancement of large behavior models that are becoming the go-to for creating AI that runs robots and robotic systems. You might not be familiar with LBMs.
No worries. I will be explaining what an LBM is, along with identifying how they end up leveraging large language models and contemporary generative AI. All told, large behavior models are quite promising and an exciting new combination consisting of LLMs boosted with behavior-oriented specialized capacities. It is a real meal deal, one might cheekily say.This analysis of an innovative proposition is part of my ongoing Forbes.com column coverage on the latest in AI including identifying and explaining various impactful AI complexities . The robot has a camera and can visually detect me and what’s throughout the kitchen. I can observe the AI. The AI can observe me. The AI is meanwhile making use of generative AI or an LLM under-the-hood. The beauty of this overall setup is that the AI can learn a task from me by observing me and asking me questions, plus, I can possibly learn from the AI by watching what it does and being inquisitive about what the AI is doing. Note that natural language is not the only element of the underway learning effort. Observations are a vital part too. Since we have an LLM amid this set up, the customary capability of computational fluency is at hand. Nice. The bonus is that the role of observations mightily ups the ante and immensely extends how things are learned. A catchy moniker for this type of AI is that we shall refer to it as being a large behavior model. This is a logical nomenclature. The AI has a similar semblance of largeness and modeling as does an LLM. The icing on the cake is that the AI can make observations and be observed and undertake and advance in training via behaviors. Just as an aside, the overall definition and notion of LBMs is still being bandied around. Some AI researchers refer to LBMs or large behavior models but might mean something different from the way I am depicting LBMs. That’s fine. Just know that the realm of LBMs is in flux and a diverse range of research endeavors are underway. Progress is happening and at some point, there will be crisper delineations of what large behavior models consist of in a globally standardized and official way.A sample dialogue of interacting with an LBM might be helpful to illustrate how things can proceed. You were earlier given a heads-up that I’m somewhat into cooking, well, not proficiently, but you get the idea. Let’s go ahead and continue the cooking theme and see what an LBM can do. Make sure to keep in mind that this same type of conversation can occur in other domains, such as a robot that works on car engines, a robot that cleans windows, a robot that guards your yard, and as many other scenarios as can be imagined. I initiated my cooking robot and opted to have the AI prepare me a meal. Yes, in a sense, my son is being replaced by a robot, but only for cooking purposes and only when he’s out-of-town. To be abundantly clear, my son is still my son, and, by gosh, no robot can ever overtake that resolute precept.“Yes. I’ve observed your preferred chopping style from previous cooking sessions. Would you like me to use a rough or fine chop this time?”“Understood. I’ll start with the carrots. Based on my multi-modal data, I estimate they should be about 1/2 inch thick for your usual texture preferences. Does that work?”“Got it! I’ll adjust my pressure and speed to match the density of each vegetable. While I chop, I will also heat the pan for the stir-fry. Pan is preheating to medium-high. Would you like me to add oil now or wait until the vegetables are chopped?”“Adding oil. I’ve learned from our past sessions that you like the vegetables to have a bit of sear. I’ll keep the heat steady and let you know when the pan reaches optimal temperature. I’m continuously learning from our sessions to refine my techniques and better assist you.” The dialogue goes on for quite a while, but I think the above excerpt is sufficient to showcase some key points about LBMs that I’ll cover next.I’m guessing you instantly recognized the interactive nature of the dialogue as being relatively representative of what happens when you converse with generative AI and LLMs. The discussion was easy going. I didn’t have to stipulate precise commands or use some quirky specialized vocabulary.I mention this noteworthy point because programming of robots has historically entailed the use of arcane coding and specialized computer programming skills. A software engineer would laboriously write lines of code to program a robot. The robot would be narrowly capable and usually respond to only a limited set of words or commands.. This makes the use of robots and their said-to-be programming a lot simpler. Happy face. But this isn’t a free lunch. There are lots of potential problems and troubles afoot. Big sad face.Suppose a robot that is roaming around your yard as a guard dog goes awry due to the generative AI encountering a so-called AI hallucination, see my coverage of such AI confabulations at. All kinds of errors and AI-related issues can arise. I’m not suggesting we avoid connecting generative AI to robots. That’s the wave of the future. Don’t think you can stop this progress. I am instead emphasizing that we need to do so mindfully, cautiously, and must weigh the ethical and legal ramifications. Period, end of story. Moving on, another aspect of the interaction involved multi-modal data. You probably are using generative AI that is based on a single mode of data, such as only conversing with you via text. Or maybe you are using an image generator that takes text and produces a nifty picture for you. I’ve been touting that we are increasingly heading toward multi-modal generative AI, see my predictions atWith LBMs, usually the AI has been data-trained in a multi-modal fashion. This contrasts with many conventional generative AI that are pretty much trained on one or two modes of data. Even if they employ multi-modes of the data, they are often doing so in a separate way and not in a fully integrated manner. LBMs gain their strengths by using multi-modal that is well-integrated, or some say the modes are fused with each other (this is somewhat like AI-based self-driving cars and multi-sensor data fusion or MSDF, see my explanation atDuring my interaction with the cooking robot, you might have subtly detected that the AI kept saying that I had been previously observed while cooking. For example, my chopping style had already been observed and the AI was data-trained on how I like to chop vegetables. The LBM then asked me if this time I wanted the AI to copy my rough chop type or my fine chop style. All in all, the keystone is that based on observations, the LBM was able to mimic my cooking regimen. I hadn’t explicitly instructed or taught the LBM how to chop vegetables, and instead I merely showed the LBM via my efforts of chopping vegetables. It was based on behavior and observations.I don’t want to seem like a gloomy Gus, but this once again has an upside plus a knotty downside. What if the AI observed me chopping vegetables and while I was doing so, I inadvertently dropped the knife? Would the AI be data trained that each time that vegetables are chopped, the knife is supposed to be dropped? That’s a real possibility of what the computational mimicry might consist of. I doubt that any adult would make that copycat mistake. Why? Partially due to common sense. It is worrisome that we do not yet have AI that somehow encompasses common sense, see my analysis at, and yet we are connecting AI to robots that move around in the physical world. For the moment, other programmatic and data training guardrails will need to serve in that safety related role.A few additional comments and then I’ll do a quick wrap-up. An AI research project that initially helped put LBM into the limelight was entitled “TRI’s Robots Learn New Manipulation Skills in an Afternoon. Here’s How.” by Siyuan Feng, Ben Burchfiel, Toffee Albina, and Russ Tedrake,“Most real-world tasks can be solved in many different ways. When picking up a cup, for example, a person might grab it from the top, the side, or even the bottom. This phenomenon,has historically been very difficult for behavior learning methods to cope with, despite its ubiquity in normal human behavior.” “Currently, robots are meticulously programmed to accomplish tasks, with humans explicitly anticipating edge cases and instructing the robot how to recover from mistakes.” “This can’t scale to the complexity required for future, more capable, robots operating in the wild.” “Existing Large Language Models possess the powerful ability to compose concepts in novel ways and learn from single examples. The next big milestone is the creation of equivalently powerful Large Behavior Models that fuse this semantic capability with a high level of physical intelligence and creativity.” “These models will be critical for general-purpose robots that are able to richly engage with the world around them and spontaneously create new dexterous behaviors when needed.”Plenty of opportunities exist in this burgeoning realm. Large behavior models are only in their infancy. The growth is going to be astronomical. We must first though iron out the kinks and resolve very tough problems. I would stridently advise AI researchers that are seeking grand challenges to give LBMs a good strong look. How can we ensure that the AI suitably identifies the right behavior? What can be done to prevent mistakes in behavioral copycatting? Are there guardrails that will on the one hand stop calamities but at the same time not unduly constrain or limit what the LBM can accomplish? Do we need new AI-related laws that will suitably govern the design, development, fielding, and use of large behavior models?Speaking of adaptability, you might know of this famous quote by Charles Darwin: “The most important factor in survival is neither intelligence nor strength but adaptability.” Making AI and especially LBMs adaptable is crucial. Let’s do so intelligently, carefully, and with assurance.Our community is about connecting people through open and thoughtful conversations. We want our readers to share their views and exchange ideas and facts in a safe space.Insults, profanity, incoherent, obscene or inflammatory language or threats of any kindContinuous attempts to re-post comments that have been previously moderated/rejectedAttempts or tactics that put the site security at riskProtect your community.
Generative AI Large Language Models LLM Large Behavior Models LBM Artificial Neural Networks Openai Chatgpt GPT-4O O1 Anthropic Claude Google Gemini Meta Llama Robot Robotics AI Ethics Law
Trending
A gorgeous April afternoon in store across the Denver metro area 
‘Artemis Mission Cannot Lead To Interplanetary Wild West,’ Astronomer Warns 
Trump says US forces will ‘finish the job’ soon in first prime-time speech since starting Iran war 
Former Wisconsin football player, who left the sport amid mental health struggles, dead at 24 
Drew McIntyre Gives Honest Take About His Recent WWE Title Reign 
U.S. Sen. Bernie Sanders introduces bill that could keep the Padres in San Diego United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Large Language Models in Oncology Not Ready for Clinical PracticeDr Maurie Markman discusses the use of large language models in oncology.
Read more »
Embracing Gen AI at WorkThe skills you need to succeed in the era of large language models
Read more »
Embracing Gen AI at WorkThe skills you need to succeed in the era of large language models
Read more »
Alpine To Use Torque Vectoring To Make Its EVs Feel LighterAlpine will also work to ensure its models are lighter than equivalent Renault and Nissan models
Read more »
AI in Medicine: Are Large Language Models Ready for the Exam Room?As large language models evolve, are they really able to assist doctors in delivering accurate, reliable information?
Read more »
HLTH ‘24: AI Models Are The Sizzle, Innovative Care Models The SteakI am the Founder and Managing Director of Summit Health Advisors, a unique strategic advisory at the intersection of healthcare and technology, specializing in multi-sided platforms.
Read more »