New AI system lets humanoid robots mimic full-body human movements in real time for precise, complex task control.
Stanford University and Simon Fraser University researchers have developed a new AI system that enables humanoid robot s to mimic human movements in real time. Known as TWIST , the technology allows users to remotely control robots by replicating their full-body motions with high precision.
Designed for real-world applications, TWIST combines motion capture data with reinforcement and imitation learning to guide robots through complex tasks. Researchers say the system marks a significant step forward in teleoperation and human-robot collaboration. “We accurately capture human motion and then use AI to map it to commands that humanoids may execute. Our system has much higher accuracy in whole-body control than prior work and enables diverse motions and skills that cannot be accomplished before,” said Yanjie Ze, first author of the paper, in a statement.Advanced teleoperation systemTeleoperation—the real-time remote control of robots—has promising applications across many fields. In recent years, engineers have focused on systems that allow users to control humanoid robots by mirroring their full-body movements. In the new research, the team aimed to enable humanoid robots to achieve the same level of whole-body dexterity as humans. In complex environments like a messy kitchen, humans naturally use their entire bodies—holding objects with both hands, moving obstacles with their feet, or opening doors with their elbows or sides. TWIST uses motion capture technology to precisely track human movements and enable real-time control of humanoid robots. By combining this data with reinforcement learning, the system trains a controller that translates human actions into balanced, full-body robot motions, allowing the robot to mimic the user’s movements closely.The system handles challenges like delays and jerky movements using a two-stage training process and combining high-quality offline data with smaller, real-time datasets. It also improves accuracy by optimizing both joint positions and orientations. The system trains robots with added disturbances to handle tasks that require force, like lifting objects, making them more stable and effective during real-world teleoperation tasks.“TWIST is a system that teleoperates humanoid robots using the whole-body movement of a person in real time,” said Ze, in a statement. Smarter humanoid robotsThe team tested their teleoperation system in real-world experiments using the G1 humanoid robot from Unitree Robotics. The system effectively enabled real-time control of the robot and showed potential for use with other humanoids, such as Booster Robotics’ T1. One key finding was that full-body human motion data alone is powerful enough to control all parts of a humanoid robot—including its legs, feet, waist, knees, and elbows—demonstrating the system’s versatility and precision.“Our system allows all these body parts to move as those of humans do, which enables robots to exhibit human-like, whole-body dexterity. The immediate application is to use humans to control humanoids to collect large-scale data to train robotic foundation models,” said Ze, in a statement. The team’s robot can perform whole-body manipulation , legged manipulation , locomotion , and expressive motions .TWIST shows promise but faces challenges, including the lack of visual and tactile feedback for operators, limited hardware durability in current humanoids, and dependence on non-portable MoCap systems. Future work aims to improve feedback, enhance hardware reliability, and adopt RGB-based pose estimation to replace MoCap for broader usability.According to researchers, with further development, the system could be deployed in real-world scenarios, enabling robots to perform manual tasks in hazardous environments or take on industrial jobs that demand high precision.“We want humanoid robots to be truly intelligent and capable of accomplishing real-world tasks. Our next step will be to scale up data collection and then to enable robots to learn autonomous skills,” said Ze, in a statement.The details of the team’s research were published in the preprint server arXiv.
Humanoid Humanoid Robot Robot Robotics Simon Fraser University Stanford University Teleoperated Whole-Body Imitation System Teleoperation TWIST
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
REAL ID appointments added as New Jersey residents struggle to get new identificationMotor vehicle bureaus add more hours to meet REAL ID demand as the new identification is required on May 7 for U.S. air travelers to fly domestically. Some New Jersey residents still struggle.
Read more »
Aligne’s New Arrivals Feature New Versions Of The Viral WaistcoatWhile Aligne made a name for its signature waistcoats and blazers, it has since become a go-to for quality denim, work and weekend essentials, and even bridal wear, which the Spring/Summer 2025 collection is full of.
Read more »
Everything revealed in the Mario Kart World Direct: A Rewind feature and moreMario Kart World adds new tricks, new course, new items, and more alongside plenty of familiarity.
Read more »
The Beths Announce North American Tour Dates Alongside New Single, ‘Metal’New Zealand indie outfit The Beths are gearing up to enter their new era, unveiling a new signing, new single, and new tour dates.
Read more »
The upcoming Sea of Stars expansion Throes of the Watchmaker arrives on May 20The eight-hour add on features a new area, new classes and a new playable character.
Read more »
New Brunswick's new cancer hospital opens a new era in care NJThe Jack & Sheryl Morris Cancer Center is one of only 13 freestanding cancer hospitals in the United States.
Read more »