Sparks of In Silico Cognitive Science: Theories from Simulated Data Can Generalize to Humans

This study tests whether cognitive theories discovered from simulated behavior can explain real human choices. Its AUTO COG system uses LLM agents to design theory-discriminating experiments, simulate responses with Centaur, arbitrate between competing executable models, and revise the weaker theory through program synthesis. The loop ran for 25 cycles in a multi-attribute decision task involving cardinal-valued cues. Centaur, a behavioral foundation model trained on trial-level data from 160 experiments, was implemented as Llama-3.1-Centaur-70B, while the discovery agents used Gemini-3.1-pro-preview; inference ran on an NVIDIA H200 GPU. The system discovered two mechanisms absent from its seed theories: Relative Contextual Salience Lexicographic, or RCSL, which combines validity-ordered search with probabilistic stopping and compensatory integration, and Contextual Relative Advantage Normalization, or CRAN, which normalizes cue differences by the strongest difference in the current context. On 10 held-out human experiments, RCSL achieved mean squared error 0.021 and CRAN 0.034, outperforming the canonical Take-The-Best score of 0.110 and Tallying score of 0.161. Their performance nearly matched theories discovered with humans in the loop. The result suggests that a simulator need not reproduce human behavior exactly: it mainly needs to preserve the distinctions that separate candidate theories. Synthetic discovery is therefore promising for broadening theory search, while human experiments remain necessary for final validation, especially when tasks move beyond the simulator’s training distribution.