Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfer
By: Haoru Xue , Tairan He , Zi Wang and more
Potential Business Impact:
Robots learn to open doors just by watching.
Recent progress in GPU-accelerated, photorealistic simulation has opened a scalable data-generation path for robot learning, where massive physics and visual randomization allow policies to generalize beyond curated environments. Building on these advances, we develop a teacher-student-bootstrap learning framework for vision-based humanoid loco-manipulation, using articulated-object interaction as a representative high-difficulty benchmark. Our approach introduces a staged-reset exploration strategy that stabilizes long-horizon privileged-policy training, and a GRPO-based fine-tuning procedure that mitigates partial observability and improves closed-loop consistency in sim-to-real RL. Trained entirely on simulation data, the resulting policy achieves robust zero-shot performance across diverse door types and outperforms human teleoperators by up to 31.7% in task completion time under the same whole-body control stack. This represents the first humanoid sim-to-real policy capable of diverse articulated loco-manipulation using pure RGB perception.
Similar Papers
VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation
Robotics
Robots learn to walk and move objects by watching.
Learning Sim-to-Real Humanoid Locomotion in 15 Minutes
Robotics
Teaches robots to walk in minutes.
From Generated Human Videos to Physically Plausible Robot Trajectories
Robotics
Robots copy human moves from fake videos.