Powered by IsaacLab
Generalizable Policy Transfer
We use a procedural generation pipeline that randomizes physical and visual properties of articulated objects: mass, handle type, hinge damping, stiffness, texture, background, etc.
We apply GRPO fine-tuning to bootstrap the student on top of classical teacher-student distillation. We find this stage to be uniquely useful for challenging loco-manipulation tasks due to partial observability. The reward is mostly binary on the task success criteria. The result: 20-30% success rate improvement.
Our policy demonstrates robust generalization to diverse real-world scenarios, successfully manipulating various door types under different environmental conditions.
DoorMan on average completes the door opening task by up to 7.15 seconds faster than human teleoperators, who struggle with skillful loco-manipulation with articulated objects.
While our policy demonstrates strong performance across diverse scenarios, we observe failure modes that highlight areas for future improvement. Common failure patterns include unobserved disturbances, distance estimation errors, and difficulties with unmodeled environmental states.
Recent progress in GPU-accelerated, photorealistic simulation has opened a scalable data-generation path for robot learning, where massive physics and visual randomization allow policies to generalize beyond curated environments. Building on these advances, we develop a teacher-student-bootstrap learning framework for vision-based humanoid loco-manipulation, using articulated-object interaction as a representative high-difficulty benchmark. Our approach introduces a staged-reset exploration strategy that stabilizes long-horizon privileged-policy training, and a GRPO-based fine-tuning procedure that mitigates partial observability and improves closed-loop consistency in sim-to-real RL. Trained entirely on simulation data, the resulting policy achieves robust zero-shot performance across diverse door types and outperforms human teleoperators by up to 31.7% in task completion time under the same whole-body control stack. This represents the first humanoid sim-to-real policy capable of diverse articulated loco-manipulation using pure RGB perception.
@misc{xue2025openingsimtorealdoorhumanoid,
title={Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfer},
author={Haoru Xue and Tairan He and Zi Wang and Qingwei Ben and Wenli Xiao and Zhengyi Luo and Xingye Da and Fernando Castañeda and Guanya Shi and Shankar Sastry and Linxi "Jim" Fan and Yuke Zhu},
year={2025},
eprint={2512.01061},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2512.01061}
}
CLAUDE.md file contains comprehensive documentation that enables coding agents to understand and adapt this website template for other projects.