Abra: Scaling Diffusion Image Training

Abra Scaling Diffusion Image Training presents a compute-optimal scaling study for text-to-image diffusion using A BRA, a controlled family of latent flow-matching transformers spanning 60M to 2B parameters and trained with µP across 1e19 to 1e22 FLOPs. Using the DataComp-1B image-text dataset, a frozen Qwen3-4B text encoder, and a FLUX VAE, the study finds that diffusion training reaches compute optimality at approximately 200 image tokens per parameter, versus Chinchilla’s 20 tokens per parameter for language models—a 10x higher data requirement. Unlike LLMs such as Chinchilla-trained models, diffusion systems tolerate overtraining: at 2× the optimal token budget, the loss penalty is less than 0.5%, so a smaller model trained on more data is usually safer than a larger undertrained model. Loss, FID, KID, CLIPScore, CMMD, and ImageNet linear-probe accuracy follow predictable scaling laws, but their optimal data-to-parameter allocations differ; optimal CFG also decreases as model size and training compute increase. The paper reports the first scaling-collapse result for diffusion training, with differently sized models’ rescaled training curves converging to the noise floor within the first 15% of training. Resolution changes the rule: estimated optimal tokens per parameter rise from 165 at 256×256 pixels to 247 at 768×768, showing that higher-resolution generation is more data-intensive in token terms. The results provide a practical scaling prescription distinct from those used for OpenAI-style language models, DeepSeek systems, or image generators built from FLUX.