HandsOn: A Large-Scale Simulation Benchmark for Generalizable Dexterous Manipulation
AuthorsTianwei Luo, Yiming He, Hongjin Wu, Ting Pan, Debin Ren, Haorui Guo, Junlin Yang, Yuxin Zuo, Zhehao Kou, Kaisen Yang, Zhuoqun Lai, Yi Ding, Jiale Yu, Yuchen Zhang, Zijie Qiu, Haozhan Li, Ning Ding
AffiliationsTsinghua University Beijing University of Posts and Telecommunications
HandsOn is a large benchmark for testing whether dexterous robots can generalize manipulation skills across hands, objects, and tasks.
Key results
Human demonstrations in the HandsOn dataset.
Core-50 and Transfer-16 human-verified tasks.
Highest success rate among trained multi-task policies.
Zero-shot prompted evaluation using move_eef.
Evaluation on held-out transfer tasks without benchmark finetuning.
What the paper found
HandsOn is a simulation benchmark for testing whether dexterous robot policies generalize across tasks, objects, and embodiments. Built in NVIDIA Isaac Sim, it supports five robotic arms, six dexterous hands, and approximately 200 assets. Its dataset contains 18000 curated human demonstrations, totaling 56 hours across three synchronized camera views, and covers 66 human-verified tasks: Core-50 provides demonstrations for training and evaluation, while Transfer-16 tests held-out asset relations, skill compositions, and deformable-object manipulation without target-task demonstrations. The benchmark also varies scene appearance and task wording while preserving recorded actions. Among trained policies, the world-action model OpenWAM achieves the highest Core-50 success rate at 29.1%. A prompted GPT-6 controller using the move_eef palm-and-finger interface, without HandsOn finetuning, reaches 5.2% on Core-50 and 1.9% on Transfer-16. Failure diagnostics show that policies often trigger tools too early, fail to coordinate both hands at once, or lose a goal state after reaching it; composed tasks add errors in step order. Real-robot rollouts suggest sim-to-real feasibility, but are qualitative rather than a scored physical evaluation.
Original abstract
General-purpose manipulation requires reliable behavior across embodiments, objects, and task demands. We introduce HandsOn, a composable simulation platform and benchmark for cross-embodiment dexterous manipulation. It supports five robotic arms, six dexterous hands, and approximately 200 rigid, articulated, and deformable assets in randomized scenes. The benchmark pairs 18,000 curated human demonstrations with 66 human-verified tasks: Core-50 supports demonstration-based learning and execution tests, while Transfer-16 evaluates asset and relation transfer, skill composition, and deformable manipulation without target-task training demonstrations. Qualitative real-robot rollouts illustrate sim-to-real transfer. Across diffusion, vision-language-action, world-action, and prompted VLM-based methods, the best trained multi-task policy achieves 29.1% Core-50 success. Without benchmark finetuning, GPT-6 using move_eef for palm-and-finger control achieves 5.2% on Core-50 and 1.9% on Transfer-16 over 660 trials. Trained-policy stage diagnostics expose failures in contact establishment, action coordination, and maintaining achieved goals, with additional sequencing errors in composed tasks. HandsOn provides a shared basis for investigating manipulation generalization and the limits of reliable physical interaction.
Read the original paperMore in AI Benchmarks
Browse all 51 papers →Last Secure Code Benchmark: Is the Code You Vibe Secure?
Tianyi Wu, Chao Ge, Peng Chen, Fengji Zhang, Yongxing Wu, Yuchong Xie, Jian Li, Chaoyuan Peng, Kaikai Zhang, Dong Huang, Mingzhe Du, Yukai Huang, Jiaheng Zhang, Terry Yue Zhuo, See-Kiong Ng
A new benchmark finds that coding agents often produce functional code that still fails to protect against real vulnerabilities.
LifeSearch: Benchmarking Deep Research Agents for Real-World Life and Productivity
Yibo Wang, Senjie Jin, Yansong Ning, Xinbing Liang, Haotian Luo, Jiazheng Zhang, Yuxuan He, Will Wu, Clarenceai
LifeSearch tests how well deep-research agents handle everyday and professional questions—and finds that even top systems still have substantial room to improve.
FrontierCS 2: A Benchmark for Computer Science R&D
Qiuyang Mang, Wenhao Chai, Runyuan He, Dongruixuan Li, Zihan Zheng, Yujun He, Xuanyi Zhou, Hao Wang, Siddhant Ray, Bo Peng, Ruyi Ji, Shuo Yang, Haocheng Xi, Boru Chen, Xinyang David Han, Melissa Pan, Mayank Mishra, Boxi Yu, Sijia Liu, Kerem Akillioglu, Yi Pan, Zirui Wang, Mert Cemri, Diogo Antunes, Yichuan Wang, Aiwei Ivy Zhang, Zhenyang Chen, Qizheng Zhang, Kaiyuan Liu, Wenyuan Huang, Sherry Ruan, Shu Liu, Dacheng Li, Zerui Cheng, Zeyu Shen, Shang Zhou, Hanfei Chen, Yuzhong Zhang, Bohan Lyu, Qu Ao, Han Zheng, Chonghe Jiang, Jingzhuo Hu, Xiaokun Chen, Kunjun Li, Zhiqing Zhong, Haoye Tian, Tong Zhao, Yunzhong He, Hanchen Li, Ofir Press, Jiantao Jiao, Kunle Olukotun, Karthik R. Narasimhan, Jingbo Shang, Koushik Sen, Matei Zaharia, Alex Dimakis, Ion Stoica, Joseph E. Gonzalez, Alvin Cheung
FrontierCS 2 tests whether AI agents can improve real research codebases across a wide range of computer science tasks.