TWIST2: Scalable, Portable, and Holistic Humanoid Data Collection System
By: Yanjie Ze , Siheng Zhao , Weizhuo Wang and more
Potential Business Impact:
Lets robots learn by watching people move.
Large-scale data has driven breakthroughs in robotics, from language models to vision-language-action models in bimanual manipulation. However, humanoid robotics lacks equally effective data collection frameworks. Existing humanoid teleoperation systems either use decoupled control or depend on expensive motion capture setups. We introduce TWIST2, a portable, mocap-free humanoid teleoperation and data collection system that preserves full whole-body control while advancing scalability. Our system leverages PICO4U VR for obtaining real-time whole-body human motions, with a custom 2-DoF robot neck (cost around $250) for egocentric vision, enabling holistic human-to-humanoid control. We demonstrate long-horizon dexterous and mobile humanoid skills and we can collect 100 demonstrations in 15 minutes with an almost 100% success rate. Building on this pipeline, we propose a hierarchical visuomotor policy framework that autonomously controls the full humanoid body based on egocentric vision. Our visuomotor policy successfully demonstrates whole-body dexterous manipulation and dynamic kicking tasks. The entire system is fully reproducible and open-sourced at https://yanjieze.com/TWIST2 . Our collected dataset is also open-sourced at https://twist-data.github.io .
Similar Papers
TWIST: Teleoperated Whole-Body Imitation System
Robotics
Lets people control robots with their whole body.
HumanoidExo: Scalable Whole-Body Humanoid Manipulation via Wearable Exoskeleton
Robotics
Teaches robots to move like humans faster.
Unveiling the Impact of Data and Model Scaling on High-Level Control for Humanoid Robots
Robotics
Teaches robots to move like humans from videos.