NTH

JustRUA: A Controlled Study of GPT-6 Robot-Use Agents

AuthorsZefan Wang, Yufan Dang, Xuanwei Shi, Mingqi Gao, Cheng Yin, SikYuen Tam, Hanyu Liu, Yasheng Wang, Yaxi Lu, Songyang Zhang, Xin Cong, Yuan Yao

AffiliationsHuazhong University of Science and Technology · Tsinghua University · Peking University · ModelBest · Tencent*Core contributor†Project lead‡Corresponding authorProject page | Code | Replays | zefan.wang.thu@gmail.com | yaoyuanthu@gmail.com

November 18, 2026 2 min read
Watch on YouTube
The one-line take

A leaner harness helps a GPT-driven robot perform much better in simulation, showing that agent design choices can matter as much as learning.

Key results

111
Evaluation episodes

Evaluation set covering all 42 RoboDojo tasks.

29.7
Official harness score

Starting evaluation-set score.

38.0
Score with 20-step motion cap

Score after retaining full episode history and capping each motion at 20 control steps.

50.3
JustRUA score

Final evaluation-set score after adding geometry inputs and coordinate-solving code.

20
Motion cap

Maximum control steps executed before the agent receives a new observation.

16
Empty grasps with predicted depth

First-round count, down from 41 without depth.

What the paper found

This controlled study tests which robot-use-agent design choices improve GPT-6 Astra on RoboDojo, using an evaluation set of 111 episodes across 42 simulated bimanual tasks. Starting from the official L3 harness, the researchers use OpenAI’s Codex to preserve the full episode history and stop each planned motion after 20 control steps; the score rises from 29.7 to 38.0. Adding robot grip readings, RGB-only depth estimates from Depth Pro, and code that derives 3D coordinates from camera pixels brings the score to 50.3, without a learned policy, task-specific tuning, or a self-improvement loop. Predicted depth cuts first-round empty grasps from 41 to 16, despite substantial distance errors, suggesting that relative depth ordering can aid manipulation. By contrast, demonstrations, self-written skills, and precomputed affordances do not clearly improve scores and add cost; experience reuse also draws on the evaluated tasks. The study concludes that a lean control harness and useful scene geometry matter more here than elaborate action interfaces, while remaining failures are largely task-specific.

Original abstract

GPT-6 Astra can drive a robot arm directly through a fixed harness, and scores of such robot-use agents on RoboDojo have risen quickly since. Reported systems differ in harness and test subset and often change several designs at once, so it is hard to tell which choices produce the gains. We take these designs apart and test them in one framework, along the harness around the model, the action interface, the reuse of experience and the scene geometry given to the model. A lean harness that keeps the full episode history and stops each motion early goes a long way. Reusing experience changes how the model acts but is not a free lunch, and scene geometry helps most, even when depth is predicted from RGB alone. JustRUA starts from the official L3 harness and keeps only what matters most. Even with no learned policy, self-improvement loop or task-specific tuning, this simple harness raises the score on the RoboDojo simulation benchmark substantially, from 29.7 to 50.3.

Read the original paper

More in Embodied AI

Browse all 69 papers →
02Embodied Ai

Kimodo-Policy: Learning Kinematic Action Models for Humanoid Loco-Manipulation

Yunheng Wang, Jiaxi Zhang, Wei Lu, Yuetong Fang, Taowen Wang, Yixiao Feng, Shuning Zhang, Junzhe Xu, Yichi Wang, Zizhao Yuan, Kailun Huang, Lusong Li, Mingqian Zhou, Zecui Zeng, Renjing Xu

Kimodo-Policy turns a pretrained whole-body motion generator into a control policy for humanoid robots that walk and manipulate objects.

Read analysis