JustRUA: A Controlled Study of GPT-6 Robot-Use Agents
AuthorsZefan Wang, Yufan Dang, Xuanwei Shi, Mingqi Gao, Cheng Yin, SikYuen Tam, Hanyu Liu, Yasheng Wang, Yaxi Lu, Songyang Zhang, Xin Cong, Yuan Yao
AffiliationsHuazhong University of Science and Technology · Tsinghua University · Peking University · ModelBest · Tencent*Core contributor†Project lead‡Corresponding authorProject page | Code | Replays | zefan.wang.thu@gmail.com | yaoyuanthu@gmail.com
A leaner harness helps a GPT-driven robot perform much better in simulation, showing that agent design choices can matter as much as learning.
Key results
Evaluation set covering all 42 RoboDojo tasks.
Starting evaluation-set score.
Score after retaining full episode history and capping each motion at 20 control steps.
Final evaluation-set score after adding geometry inputs and coordinate-solving code.
Maximum control steps executed before the agent receives a new observation.
First-round count, down from 41 without depth.
What the paper found
This controlled study tests which robot-use-agent design choices improve GPT-6 Astra on RoboDojo, using an evaluation set of 111 episodes across 42 simulated bimanual tasks. Starting from the official L3 harness, the researchers use OpenAI’s Codex to preserve the full episode history and stop each planned motion after 20 control steps; the score rises from 29.7 to 38.0. Adding robot grip readings, RGB-only depth estimates from Depth Pro, and code that derives 3D coordinates from camera pixels brings the score to 50.3, without a learned policy, task-specific tuning, or a self-improvement loop. Predicted depth cuts first-round empty grasps from 41 to 16, despite substantial distance errors, suggesting that relative depth ordering can aid manipulation. By contrast, demonstrations, self-written skills, and precomputed affordances do not clearly improve scores and add cost; experience reuse also draws on the evaluated tasks. The study concludes that a lean control harness and useful scene geometry matter more here than elaborate action interfaces, while remaining failures are largely task-specific.
Original abstract
GPT-6 Astra can drive a robot arm directly through a fixed harness, and scores of such robot-use agents on RoboDojo have risen quickly since. Reported systems differ in harness and test subset and often change several designs at once, so it is hard to tell which choices produce the gains. We take these designs apart and test them in one framework, along the harness around the model, the action interface, the reuse of experience and the scene geometry given to the model. A lean harness that keeps the full episode history and stops each motion early goes a long way. Reusing experience changes how the model acts but is not a free lunch, and scene geometry helps most, even when depth is predicted from RGB alone. JustRUA starts from the official L3 harness and keeps only what matters most. Even with no learned policy, self-improvement loop or task-specific tuning, this simple harness raises the score on the RoboDojo simulation benchmark substantially, from 29.7 to 50.3.
Read the original paperMore in Embodied AI
Browse all 69 papers →Demonstration of MANA: Multi-perspective Alignment of NPC Agents
Amir Tahmasbi, Keshav Singh, Shaheer Raza, Owen Mei, Jason Wu
MANA lets game developers step into an NPC’s shoes and demonstrate how it should behave, then lets the character act on its own.
Kimodo-Policy: Learning Kinematic Action Models for Humanoid Loco-Manipulation
Yunheng Wang, Jiaxi Zhang, Wei Lu, Yuetong Fang, Taowen Wang, Yixiao Feng, Shuning Zhang, Junzhe Xu, Yichi Wang, Zizhao Yuan, Kailun Huang, Lusong Li, Mingqian Zhou, Zecui Zeng, Renjing Xu
Kimodo-Policy turns a pretrained whole-body motion generator into a control policy for humanoid robots that walk and manipulate objects.
One Policy for 100 Household Tasks: Our Solution for the 2026 BEHAVIOR Challenge
Ilia Larchenko, Gleb Zarin, Atindra Nair, Keerti Manney, Vitaly Kleban, Mirko Kiefer, Anjana Yodaiken
A single vision-language-action policy tackles 100 simulated household tasks, with a coding agent stepping in for some of the hardest ones.