NTH

Ask, Don't Equip: Question-Driven Coding Agents for Deep Kernel Optimization

AuthorsGabriele Oliaro, Jaeseong Lee, Zhaoyuan Su, Sreeram Vennam, Zhihao Jia, Samyam Rajbhandari

November 7, 2026 3 min read
Watch on YouTube
The one-line take

KernelSeek shows how asking an AI coding agent the right question can help it build faster GPU kernels without handing it the answer.

Key results

0.50
Autonomous-agent speedup

Geometric-mean speedup over the seed-relative FA3 benchmark after 133 model-driven steps.

1.13
KernelSeek head-to-head speedup

Geometric-mean speedup over FlashAttention-3 on the same 26 configurations.

1.16
Hopper FA3 speedup

Geometric-mean speedup over FlashAttention-3 across 182 Hopper scenarios.

1.46
Blackwell FA4 speedup

Geometric-mean speedup over FlashAttention-4 on paired Blackwell scenarios.

154
Runnable Blackwell scenarios

Scenarios on which the Blackwell kernel was correct.

101
Core question bank

Reusable questions designed to direct measurements without prescribing solutions.

What the paper found

KernelSeek replaces tool-heavy, autonomous kernel search with a general-purpose coding agent, a minimal harness that checks correctness against the seed and an fp32 reference, and short questions posed when progress stalls or the agent claims a limit. The questions provide no answers: they prompt measurements and redirect attention across the kernel, host-launch path, routing, and benchmark. A reusable bank contains 101 core questions, and the agent can apply them to itself. In a controlled comparison using Claude Opus on NVIDIA H200 hardware, a tool-rich autonomous agent plateaued at 0.50× FlashAttention-3 speed after 133 model-driven steps, while KernelSeek reached 1.13× on the same 26 configurations. In full campaigns, its dependency-free CUDA kernels—using techniques such as warp specialization, asynchronous tensor-core operations, and host-side routing—achieved 1.16× FlashAttention-3 on Hopper across 182 scenarios and 1.46× FlashAttention-4 on Blackwell; the Blackwell kernel was correct on 154 runnable scenarios. The trace analysis found that large gains followed questions that changed which system layer the agent examined or challenged an assumed ceiling, not questions that supplied new technical information. The results suggest that directing an agent’s measurements can matter more than adding tools or reference material.

Original abstract

LLM agents for GPU kernel optimization sit at two ends of a structure-freedom tradeoff. Scripted agents fix what the model does at each step, via a learned policy, an optimization menu, or fixed roles, and are sample-efficient where the recipe applies and stall where it does not. Open-ended agents let a self-directed model mutate a population of kernels and pay for the freedom in trials: thousands of evaluations, repeated edits, and wrong candidates. Both respond to hard targets by equipping the agent with more: parsed profiler output, disassembly analysis, retrieval over documentation and libraries, and stores of lessons. We argue that both lack direction, a hypothesis about where the remaining gap is, and that a short question asked at the right moment supplies it without narrowing what the agent may do next. We present KernelSeek, a question-driven agent that makes an existing CUDA kernel faster with no operator-specific knowledge and no faster implementation to imitate: a general-purpose coding agent, a minimal harness that enforces a verifiable goal, and questions at decision points, first from a human, then from a bank the agent applies to itself. On the same task, evaluator, hardware, and week, a tool-rich autonomous agent built in the prevailing style plateaued at 0.50x FlashAttention-3 after 133 model-driven steps; KernelSeek reached 1.13x. It then produced dependency-free FlashAttention forward kernels for Hopper and Blackwell, single CUDA files with no CUTLASS, CuTe, cuBLAS, or cuDNN, correct on all 182 scenarios per architecture and 1.16x FlashAttention-3 and 1.46x FlashAttention-4 (geometric mean). Across 162 human messages and 11.5K agent tool calls, every large gain followed a question that moved the agent to a different layer of the system or rejected a claimed limit, never one that supplied information. We release the kernels, harness, question bank, and all traces.

Read the original paper

More in Code Generation

Browse all 52 papers →
01Code Generation

CompoundingKernel: Self-Improving from Zero to First

Qian Qu, Hongyang Wang, Wenbo Chen, Yichuan Li, Jingguo Liu, Xian Li, Xuan Wang, Ruochen Wen, Leo Fan, Yiming Yan, Zheshen Wang, Benoit Dumoulin

CompoundingKernel uses parallel agents and an archive of successful attempts to discover fast, correct GPU kernels.

Read analysis