NTH

CompoundingKernel: Self-Improving from Zero to First

AuthorsQian Qu, Hongyang Wang, Wenbo Chen, Yichuan Li, Jingguo Liu, Xian Li, Xuan Wang, Ruochen Wen, Leo Fan, Yiming Yan, Zheshen Wang, Benoit Dumoulin

AffiliationsAmazon

November 20, 2026 2 min read
Watch on YouTube
The one-line take

CompoundingKernel uses parallel agents and an archive of successful attempts to discover fast, correct GPU kernels.

Key results

235
Site-verified correct problems

Every SOL-ExecBench problem had a kernel verified correct by the site.

4.40×
Public-problem speedup

Geometric-mean speedup over the benchmark baseline across 209 public problems.

35
Controller wins

Problems won out of 38 paired comparisons against the coded UCB-style tree.

2.4×
Correct-attempt rate advantage

The controller produced this many correct attempts in equal wall time.

3.6×
Evaluation throughput increase

Candidates evaluated per hour after a warm-restart change.

What the paper found

Amazon’s CompoundingKernel uses a fixed Claude Opus 5.5 model to search for GPU kernels on NVIDIA B200s, starting from PyTorch definitions without access to reference kernels or official scores. Its key design is a per-problem archive that preserves every correct kernel and its lineage: an LLM controller selects a parent and gives each parallel child agent one optimization direction, while external checks verify correctness and measurements. On SOL-ExecBench, all 235 problems received site-verified correct kernels; on the 209 public L1, L2, and Quantization problems, the kernels achieved a 4.40× geometric-mean speedup over the benchmark baseline and ranked first on all three boards. In a comparison with a coded UCB-style tree, the controller won on 35 of 38 paired problems, mainly by producing 2.4× as many correct attempts in equal time, rather than choosing better individual attempts. At fleet scale, a warm-restart change that improved evaluation throughput raised measured candidates per hour by 3.6×. The results point to reliable, fast evaluation—not simply adding agents—as the constraint on scaling kernel search.

Original abstract

We present CompoundingKernel, a self-improving agent system that writes GPU kernels by search. As of 2026-11-11, its generated kernels rank #1 on the public L1, L2 and Quantization boards of SOL-ExecBench (NVIDIA B200). They have held #1 on L1 since 2026-10-27, albeit a short loss on 2026-11-07. All 235 problems have a site-verified correct kernel; on the 209 public-board problems they are 4.40x faster than the benchmark's reference baseline. The search starts from zero: agents get each problem's PyTorch definition but no reference kernels or official scores. A per-problem archive outside the model holds every correct kernel and its lineage. An LLM controller reads it, picks a parent or a new design, and gives each parallel child agent one direction. Checks outside the model decide correctness. At equal wall time, it beat a coded UCB style tree on 35 of 38 problems by making 2.4x as many correct attempts. Over a thousand agents ran at once, and one warm restart fix raised their evaluation rate 3.6x. We find that the rate of reliable measurements, not the number of agents, limits how far such a search scales. Our code and kernels are available on URL.

Read the original paper

More in Code Generation

Browse all 52 papers →