Test-Time Compute Scales Better with Parallel Agent Teams
AuthorsJunlin Wang, Haozhe Chen, Ang Li, Rahul Chalamala, Zhuang Liu
Affiliations[
Instead of choosing between many solo agents and one huge team, run many small teams in parallel and keep the strongest result.
Key results
Parallel teams of ten versus parallel solo agents, averaged across 12 tasks.
Parallel teams of ten versus parallel solo agents, averaged across 12 tasks.
Parallel teams beat parallel solo agents on 10 of 12 tasks.
At 100 agents, parallel teams beat one team of 100 by 8% to 30% on all four tested tasks.
Qwen3.8-27B parallel teams beat GPT-5.6-sol on 4 of 8 tasks.
What the paper found
This paper introduces parallel agent teams: instead of spending an inference budget on many solo agents or one oversized team, it runs several small teams independently and uses a verifier to select the best result. Each flat team shares findings through a forum, without assigned roles or a manager. Across 12 long-horizon tasks, Qwen3.8-27B teams of ten improved average scores over solo-agent sampling by 10% at a 10-agent budget and 27% at 100 agents, beating that baseline on 10 of 12 tasks. At 100 agents, parallel teams also beat a single team of 100 by 8% to 30% on all four tasks tested. The explanation is a mean-variance trade-off: collaboration raises each sample’s average quality, while sampling multiple teams preserves the benefits of variation. A single large team sacrifices that sampling benefit, while solo agents produce more samples but with lower average scores. At equal dollar cost, Qwen3.8-27B teams beat OpenAI’s GPT-5.6-sol on 4 of 8 tasks; tests with DeepSeek V4.1 Flash show that team gains are smaller on tasks the model already handles well. The results were produced using the OpenAI Codex harness, and suggest that parallel collaboration can make added test-time compute more effective when candidate solutions can be verified.
Original abstract
Test-time scaling improves the performance of a fixed model by spending more computation at inference. There are two common ways to spend it on agents when a verifier can score every attempt. Best-of-N sampling runs many independent agents and keeps the best result but saturates quickly. One large team of collaborating agents can score higher but stops improving as it grows. Can we get the best of both? Our method, parallel agent teams, does so by sampling many small teams in parallel and keeping the best team's result. We test it on 12 long-horizon tasks from GPU kernels and formal proofs to exploit development. With a 27B model and a budget of 100 agents, parallel teams of ten beat parallel agents on 10 of 12 tasks and beat one team of 100 on all four tasks where we ran such large teams. At equal dollar cost the same parallel teams beat a frontier model on half the tasks. A simple statistical model shows that individual agents lose because their many samples have a low mean and that one large team loses because a single sample gets no gain from variance. Sampling many small teams in parallel balances a high mean with the gain from variance. Parallel agent teams thus offer a more effective way to scale test-time compute.
Read the original paperMore in AI Agents
Browse all 77 papers →From Attack Surface to Defense Architecture: Compositional Security for Agentic Coding Harnesses
Simon Hegelich, Bardh Prenkaj, Habiba Sarhan, Morteza Shahrezaye, Kolja Hegelich, Christian Montag, Ralf Schneider, Erik Cambria
A security architecture aims to keep coding agents useful while preventing malicious instructions from turning their tools into an attack platform.
HarnessFlow: MCTS-Guided Harness Optimization for LLM Agents
Zhi Hong, Qianlong Wang, Hongchen Wei, Aidan Wu, Daohong Ying, Mingze Kong, Bei Liu, Chong Luo
HarnessFlow uses tree search and a coding agent to automatically refine the software harnesses that guide LLM agents.
Experience-Handoff Reinforcement Learning for Recursive Self-Improvement
Doyoung Kim, Byeongguk Jeon, Kailin Zhang, Sherry Yang
The work trains an AI agent not only to act, but also to write useful playbooks that help its future attempts succeed.