Learning High-Frequency Continuous Action Chunks in Latent Space

Learning High-Frequency Continuous Action Chunks in Latent Space argues that robotic action chunking breaks down at 60 Hz when policies predict directly in action space, producing jitter, quantization error, and boundary stalls. The paper’s core idea is to encode 48-step high-frequency action chunks with a variational autoencoder, train the policy in a 10-dimensional latent space, and decode back to actions at execution time; this consistently improves trajectory smoothness and precision across Diffusion Policy, OpenVLA-OFT, and PI0.5. On real-world contact-rich tasks—Peel Cucumber, Wipe Vase, and Write Board—the latent formulation reduces jerk sharply; for example, OpenVLA-OFT on Write Board drops from 5.238 to 0.558 jerk and lifts success from 74% to 100%, while Diffusion Policy on the same task drops from 1.140 to 0.511 jerk. To handle asynchronous inference, the authors add Reuse-then-Refine, a training-free method that reuses overlapping executed actions, concatenates them with the new chunk, and refines the result through the VAE; this lowers chunk-boundary gaps and reduces execution stalls, especially for PI0.5 where the asynchronous write-board jerk falls from 4.984 to 1.754. The experiments also show that moderate latent compression helps, but overly aggressive compression degrades both precision and smoothness, and the VAE adds only about 2.30 ms of encode-decode overhead.