Resources
AEGIS helps robots avoid failure spirals by predicting trouble early and handing control to a stronger policy only at the risky moments.
Key results
common-random-number episodes per arm on LIBERO-Spatial
A-failing trajectories in the confirmatory run
failure probe performance over the first 30% of trajectory steps
conditional recovered-task rate on weak-policy failures
budget-matched blind escalation baseline
random-trigger placebo baseline
What the paper found
AEGIS, from KAIKAKU, is a runtime “backup reflex” for physical AI that reads a frozen robot policy’s own activations to predict imminent long-horizon manipulation failure, then selectively hands control to a stronger separate policy before the trajectory collapses. Built on the LIBERO benchmark with a weak SmolVLA 450M policy and a stronger π0.5/π0.5 finetuned 4.14B executor, the method uses a two-layer [720 → 256 → 1] probe on layer-15 action-expert activations, calibrated with split conformal prediction at a nominal 0.10 per-step false-trigger rate, plus an early-harm gate at 0.20T and a per-episode cap of 0.05T escalations. On the confirmatory LIBERO-Spatial run with 700 common-random-number episodes and 646 weak-policy failures, the early-window failure probe reaches AUROC 0.764 over the first 30% of trajectory steps, and AEGIS recovers 10.1% of failed trajectories versus 4.6% for budget-matched blind escalation and 5.1% for a random-trigger placebo, with Holm-adjusted exact paired tests of 8.5×10−6 and 1.0×10−4. The stronger policy is active on only 38% of steps, yet the method roughly doubles matched-budget recovery while preserving a 6.5 recover-to-disrupt ratio, demonstrating that timing—not extra compute—drives the gain. The paper also shows the effect survives difficulty stratification and replicate-resampling robustness checks, and generalizes when the escalation target is swapped to NVIDIA’s GR00T N1.x.
Original abstract
Long-horizon robot manipulation tends to fail gradually: one bad step degrades the state, and the policy spirals into a basin from which it cannot recover. The failure is often visible before it happens. We introduce AEGIS (Activation-probe Early-warning, Gated Inference Switching), a selective escalation method that uses a lightweight probe on a weak policy's frozen activations to detect high-risk steps while there is still time to act. When the probe flags a step, control switches to a stronger separate policy, but only for the steps that need it. On LIBERO-Spatial, AEGIS recovers 10.1% of the trajectories the weak policy alone loses, versus 4.6% for budget-matched blind escalation and 5.1% for a random-trigger placebo. These gains are significant under one-sided exact paired McNemar tests with Holm-Bonferroni adjustment over three pre-registered contrasts: +5.4pp over blind escalation, p=8.5e-6; +5.0pp over random triggering, p=1.0e-4; paired-trajectory bootstrap CIs exclude zero. AEGIS activates the stronger policy on only 38% of steps, so the lever is timing rather than compute. The probe clears its precondition with an early-window AUROC of 0.764, 95% CI [0.70, 0.84], read from the weak-policy path over the first 30% of trajectory steps before any handoff. We pre-register the full analysis plan, including a conditional recovered-task-rate estimand and explicit kill criteria, and confirm the result on 700 common-random-number episodes per arm, with nA-fail=646.
Read the original paperMore in Robotics
Browse all 50 papers →JAMB: Joint Action-Motion Diffusion for Bimanual Manipulation
Chuyang Xiao, Peilin Meng, David Held
JAMB helps two robot arms coordinate by jointly imagining their future movements and the changing 3D scene before acting.
Rolling-WAM: World Action Models with Rolling Imagination
Yinghua Zhou, Junjie Ye, Yiqi Zhao, Hao Dong, Celina Shiyu Wang, Ruohai Ge, Tingyi Yang, Basile Van Hoorick, Gaurav Sukhatme, Vitor Guizilini, Yue Wang
Rolling-WAM keeps future robot actions partially imagined and refined over time, making world-model-based manipulation replan 4.5 times faster.
Training-free Behavior Cloning
Maximilian Adang, Timothy Chen, Lars Osterberg, Aiden Swann, Mac Schwager
A fast, training-free robot controller reuses and corrects demonstration trajectories to deliver traceable behavior at real-time speeds.