EdgeBench: Unveiling Scaling Laws of Learning from Real-World Environments

EdgeBench from ByteDance Seed introduces a benchmark for studying post-deployment learning in real-world environments, spanning 134 executable tasks across scientific problems and ML, systems and software engineering, combinatorial optimization, professional knowledge work, formal math, and games. Across roughly 38,000 hours of agent interaction, the paper reports that aggregate best-so-far performance over time follows a highly precise log-sigmoid law, S(t)=Smax/(1+(tmid/t)^β), with mean R²=0.998 over the full 134-task average and R²≥0.993 even when extending trajectories to 28 and 72 hours. The same functional form fits each task family, outperforms log-probit, log-Gompertz, Weibull, and log-linear baselines, and can forecast 12-hour performance from only the first 6.5 hours with R²≥0.997 and RMSE below 1.0. The authors also argue for a frontier-expansion mechanism on latent task graphs, where accumulated unlocked capability and remaining locked opportunity drive sigmoid growth in log time. On the model side, learning speed roughly doubles every 3 months across frontier systems, rising by about 8× from GPT-5-Codex in September 2025 to GPT-5.5 in April 2026. In the 12-hour leaderboard, Claude Opus 4.8 leads overall with 51.3, followed by GPT-5.5 at 48.4, and a stateful 12-hour run on the gravitational-wave reconstruction task reaches 67.0 after 247 scored evaluations.