MIT’s CSAIL researchers developed a system that lets soft robots learn how their bodies move using only vision and AI with no sensors, models or manual programming.
bodies, using only vision. Instead of relying on sensors, the system allows robots to learn how their bodies move and respond to commands just by watching themselves.that teaches robots to understand their bodies, using only vision. Using consumer-grade cameras, the robot watched itself move and then built an internal model of its geometry and controllability.
“We use motion tracking algorithms - point tracking and optical flow - to track the motion of the robot during training,” said Sitzmann. "By relating the motion of the robot to the commands that we instructed it with, we reconstruct our proposed Neural Jacobian Field, which endows the 3D model of the robot with an understanding of how each 3D point would move under a particular robot action.
A soft pneumatic hand learned which air channel controls each finger, not by being told, but just by watching itself wiggle. They inferred depth and geometry from color video alone, reconstructing 3D shape before and after actions. “However, your joints also aren’t perfectly rigid like those of a robot, they can similarly bend and give in, and while you can sense the approximate position of your joints, your highest-precision sensors are vision and touch, which is how you solve most manipulation tasks,” said Sitzmann. "Soft robots are inspired by these properties of living creatures to be similarly compliant, and must therefore necessarily also rely on different sensors than their rigid cousins.
MIT CSAIL Research Innovation Neural Networks AI Computer Vision Technology Science
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Shape-shifting soft robots deliver real human touch with 16 powerful haptic modesShape-shifting soft robots deliver customizable vibrations and stiffness, opening new possibilities in rehab and immersive tech.
Read more »
MIT’s new AI outwits scientists to design robots that jump 41% higher, land safelyIn robot development, AI-generated linkages are curved whereas the standard robot’s connecting parts are straight and rectangular.
Read more »
Humanoid robots could soon see through walls with MIT’s new imaging breakthroughDuring tests, the MIT team's robotic radar system achieved 96 percent when reconstructing hidden and obscured objects.
Read more »
Jennifer Lawrence Teaches Pajama Dressing 101 in a Lacy Nightgown, Slippers, and Prada BagMeguire Hennes is the fashion staff writer at Marie Claire, where she breaks down the celebrity looks living rent-free in her head (and yours). Whether a star is walking the red carpet or posing on Instagram, Meguire will tell you who they're wearing and why. When she's not gushing about A-listers from J.
Read more »
Sex tech teaches our kids about sex long before we do.If you’re not talking to your kids about sex, someone—or something—already is.
Read more »
Opinion: Juneteenth in Alaska is more than a holiday — it’s a call to actionIt’s not enough to gather — we need to act on what Juneteenth teaches.
Read more »