HumanCLAW: Can Vision-Language Models Act Through a Body?

HumanCLAW, developed by researchers at Meta, Nanyang Technological University, and the University of Washington, asks whether vision-language models can make reliable moment-to-moment decisions through a humanoid body. Its key contribution is a decoupled evaluation loop: a frozen, off-the-shelf VLM selects one atomic skill, such as walking or turning, a skill-conditioned motion generator converts it into continuous full-body motion, and a half-physics simulator preserves gravity, collisions, contact, and object displacement while removing balance and motor-tracking failures. The resulting HumanCLAW-Bench contains 1,218 egocentric find-navigate-interact episodes across 41 indoor scenes, evaluated with nine VLMs including Gemini-3.1, GPT-5.5, Claude-4.8, and Gemma-4-31B. None solves the benchmark; Gemini-3.1 achieves the best complete-task success at 16.8%. The models generally recognize targets once they appear, but fail to estimate their own position, detect arrival, notice collisions, and place the body correctly for sitting—evidence of missing embodied self-awareness rather than inadequate object recognition. The motion layer uses a 38M-parameter, 10-layer diffusion transformer trained on AMASS motion chunks, with plug-and-play per-skill ControlNet adapters; commanded walking achieves an execution ratio of 0.966. A verifier is crucial: on a 100-episode ablation, removing it reduces navigation success from 27.0% to 2.0%, showing that compact, skill-specific checks can stabilize closed-loop action without retraining the VLM.