Loop the Loopies!

“Loop the Loopies!” from IQuest Research introduces Loopie, a pair of Qwen3-MoE-style language models that make recurrent computation competitive with conventional Transformer scaling under matched pre-training time. Loopie-20B-A2B contains 20B total parameters with 2B active per token, while Loopie-6B-A0.6B contains 6B with 0.6B active. Its central innovation is layer-loop recurrence: each attention/MoE layer is applied twice consecutively before the model advances, rather than repeatedly traversing the entire stack as in model-loop systems such as Ouro and Huginn. The hardware-aware Loopie Recipe halves stored depth, doubles the per-device microbatch, and reinvests the measured efficiency gain into width and capacity; the final model reaches 261.53 TFLOPS/s peak throughput versus 189.65 TFLOPS/s for a Qwen3-like 30B-A3B baseline. In matched training, Loopie overtakes that vanilla baseline after roughly 600B tokens. Pre-training uses NVIDIA’s Nemotron-CC-v2-HQ and related Nemotron datasets, followed by 2T tokens of supervised pre-training at large language-model batch scales, then reinforcement learning using GSPO with DAPO-style asymmetric clipping and dynamic filtering. The resulting Loopie-20B-A2B Thinking scores 92.09 on AIME 2024 and 94.21 on AMC, after only 3.5T pre-training tokens compared with 25T for the cited Nemotron models. The paper’s main conclusion is that carefully scheduled recurrence, rather than looping alone, can become a scalable compute-allocation strategy for sparse large language models.