Scaling Cross-Embodiment World Models for Dexterous Manipulation
Researchers introduce a method for representing both human and robot hands as sets of 3D particles, with actions defined as end-effector particle displacement fields, to facilitate cross-embodiment learning. They train a graph-based world model on diverse simulated and real hand data, and show that this model, combined with model-predictive control, enables effective dexterous manipulation on previously unseen robotic hands with different kinematics. The study demonstrates that increasing the diversity of training embodiments improves generalization, and that mixing simulated and real data yields better performance than using either alone.
Why it matters: This work provides a shared geometric representation that could accelerate the development of generalist robots capable of learning manipulation skills across a wide range of hand morphologies.
Full story at: arXiv Robotics ↗