Interpretable weight-space programming builds transparent transformers and brain encoding models
AuthorsYash Mehta, Chandan Singh, Weijia Xu, Richard J. Antonello, Adeen Flinker, Nima Mesgarani, Michael F. Bonner, Nebojsa Jojic, Jianfeng Gao
AffiliationsMicrosoft Research · Johns Hopkins University · Columbia University · New York University github.com/csinva/weight-space-programming-autoresearch
An LLM agent hand-builds interpretable neural networks that predict brain activity and suggest what computations the brain may be using.
Key results
Claude Opus 5.5 achieved this on its algorithmic task runs.
Smallest algorithmic models found by Claude Opus 5.5.
iWSP prediction on the Natural Scenes Dataset; best interpretable baseline scored 0.400.
iWSP prediction on the Natural Scenes Dataset; best interpretable baseline scored 0.256.
How closely story-position features tracked the scan-timing response pattern.
Increase in hidden-set correlation over Whisper alone, from 0.070 to 0.075.
What the paper found
Interpretable weight-space programming, or iWSP, asks an LLM agent to write neural-network weights and features directly in code, then revise them using evaluation scores—without backpropagation or training. Separate audits and tests that rebuild models from short descriptions check whether the resulting computations are genuinely interpretable. Using Anthropic’s Claude Opus 5.5 and Gemini 4 Carbon, the method produced transformers that reached 100% hidden accuracy on algorithmic tasks; the median smallest model reaching perfect visible accuracy used 166 parameters. For visual brain encoding, its 14 explicit computations outperformed interpretable baselines on the Natural Scenes Dataset, reaching correlations of 0.443 in early visual cortex and 0.317 in ventral visual cortex. In story-listening fMRI, iWSP exposed scan-position features that tracked an unintended timing signal at 0.97 correlation, pointing to the need to trim scan edges. For auditory ECoG, its readable acoustic features complemented Whisper: combining them raised hidden-set correlation from 0.070 to 0.075, a gain of 0.0052. The paper also compares language-fMRI predictions with GPT-2 XL. Overall, iWSP’s contribution is a practical way to search for predictive models whose internal computations can be inspected, tested, and reconstructed—while its results still rely on known computational motifs rather than new ones.
Original abstract
Neural networks are powerful predictors, yet their opacity limits their usefulness in science, where the goal is explanation rather than mere prediction. In neuroscience, for example, deep encoding models accurately predict brain activity in response to natural stimuli, but struggle to reveal which stimulus features drive those predictions. Here we introduce interpretable weight-space programming (iWSP), a method in which an LLM agent iteratively builds interpretable models by explicitly programming their weights and features in code, without gradient-based training (backpropagation). Without backpropagation to tune parameters, the agent must draw on its prior knowledge and reason over previous trials to hypothesize, hand-craft, and refine human-interpretable computations mirroring the workflow of a human scientist. LLM audits confirm the interpretability of the hand-crafted models. We first validate iWSP by tasking it with constructing transformers for canonical benchmarks such as arithmetic and sentiment classification, finding that LLM agents can successfully craft compact, accurate networks directly. Next, we apply iWSP to build neural encoding models across vision, language, and audition. In the visual cortex, iWSP outpredicts existing interpretable baselines, uncovers distinct computations across early/higher visual areas, and generalizes to unseen data. In language fMRI, iWSP surfaces a confound in the data and in auditory ECoG, it systematically discovers interpretable acoustic features that predict neural activity. Together, these results establish iWSP as a general paradigm for aligning high predictive performance with human-interpretable scientific discovery.
Read the original paperMore in AI for Science
Browse all 58 papers →DrugCoding: A Scientific Programming Agent for Drug Discovery
Junwei Hu, Zhuohao Lin, Kun Li, Hengnan Zhang
DrugCoding is an AI agent that plans and executes multi-step computational tasks to help automate drug discovery.
Reliability by Architecture: Agentic Workflows for Validated Multi-Physics Simulation from Natural Language
Søren Taverniers, Eric Darve, Hamdi A. Tchelepi
An LLM-assisted workflow prepares and repairs multi-physics simulations, while deterministic checks help ensure the results match both the physics and the user’s request.
Sample-Efficient Control of Magnetoconvective Flows via Differentiable Simulation
Jannis Becktepe, Dmitry Krasnov, Nils Thuerey, Jörg Schumacher, Sebastian Peitz
A differentiable simulator helps researchers control magnetoconvective flows more efficiently and design better heat-transfer outcomes.