DualTHOR: A Dual-Arm Humanoid Simulation Platform for Contingency-Aware Planning
By: Boyu Li , Siyuan He , Hang Xu and more
Potential Business Impact:
Teaches robots to use two arms to do tasks.
Developing embodied agents capable of performing complex interactive tasks in real-world scenarios remains a fundamental challenge in embodied AI. Although recent advances in simulation platforms have greatly enhanced task diversity to train embodied Vision Language Models (VLMs), most platforms rely on simplified robot morphologies and bypass the stochastic nature of low-level execution, which limits their transferability to real-world robots. To address these issues, we present a physics-based simulation platform DualTHOR for complex dual-arm humanoid robots, built upon an extended version of AI2-THOR. Our simulator includes real-world robot assets, a task suite for dual-arm collaboration, and inverse kinematics solvers for humanoid robots. We also introduce a contingency mechanism that incorporates potential failures through physics-based low-level execution, bridging the gap to real-world scenarios. Our simulator enables a more comprehensive evaluation of the robustness and generalization of VLMs in household environments. Extensive evaluations reveal that current VLMs struggle with dual-arm coordination and exhibit limited robustness in realistic environments with contingencies, highlighting the importance of using our simulator to develop more capable VLMs for embodied tasks. The code is available at https://github.com/ds199895/DualTHOR.git.
Similar Papers
Towards Proprioception-Aware Embodied Planning for Dual-Arm Humanoid Robots
Robotics
Helps robots with two arms follow complex instructions.
Thor: Towards Human-Level Whole-Body Reactions for Intense Contact-Rich Environments
Robotics
Robots can push and pull like people.
Thor: Towards Human-Level Whole-Body Reactions for Intense Contact-Rich Environments
Robotics
Robot learns to push and pull like a person.