AnyMo: Scaling Any-Modality Conditional Motion Generation with Masked Modeling

AnyMo, from researchers at the Chinese Academy of Sciences and independent collaborators, tackles conditional human motion generation with a masked-modeling design that scales to arbitrary combinations of text, speech, music, and trajectory inputs. Its foundation is OmniHuMo, a new web-video-derived dataset with over 5,000 hours and 3.2 million motion sequences, each paired with text and a subset of about 500 hours also aligned with audio. The model replaces single-stream tokenization with a 4-layer Residual FSQ tokenizer using 2048 codes per layer, then trains a LLaMA-based bidirectional Transformer with parallel masked prediction across residual token streams. This lets AnyMo model global temporal context while conditioning on multiple modalities in a shared embedding space. On OmniHuMo-Text, scaling the model from 111M to 3B parameters steadily improves FID from 262.10 to 55.59 and raises R-Precision top-1 from 0.63 to 0.75. In multimodal conditioning, adding trajectory input cuts text-guided average trajectory error from 50.50 cm to 27.16 cm and lowers the share of errors above 50 cm from 0.52 to 0.33. The tokenizer also benefits from scale: reconstruction MPJPE drops from 94.55 mm at 0.05M data to 27.92 mm at 3M. Overall, the paper argues that large-scale multimodal alignment, not just larger models, is the key driver of flexible motion synthesis.