EnigmaForge: The Question Is Hidden in the Story

EnigmaForge evaluates whether language models can discover a problem before solving it: each instance provides only a procedurally generated story assembled from letters, receipts, and logbook fragments, with no stated question. The model must infer the hidden task, reconstruct a finite-domain constraint world, use external knowledge bridges, and execute a policy-validated action. Generation begins with a known solution, strengthens and minimizes its evidence, then verifies uniqueness with a DPLL SAT solver, brute-force cross-checking, and per-clue ablation certificates showing that removing any clue creates a second solution. Seed-based generation supports renewable, contamination-resistant evaluation, with multiple prose realizations across five genres and six difficulty levels. Across 600 instances, 120 families, and 17,400 scored records, 25 frontier models and four deterministic baselines were tested in explicit-question, formal-world, and implicit-story conditions. The results separate reconstruction from intuition: unguided task success ranges from 3.6 to 80.1, a 22-fold spread, while fact F1 spans only 1.6-fold. OpenAI's GPT-6 Sol achieves the strongest fact F1 at 0.966 and 75.0 implicit task success; OpenAI's GPT-6 Astra leads intuition at 80.1. Anthropic's Claude Opus 5.5 and DeepSeek-V4 Pro reconstruct worlds well but rank far lower on intuition, while Google's Gemini models generally lose more performance as difficulty rises. xAI's Grok-4.6 shows a 0.000 explicit-to-implicit task gap, whereas GPT-6 Sol performs significantly better without the question, at −0.142. Content filters also distort measurement: Claude Fable 5.1 blocks 585 of 600 attempts, demonstrating that refusal handling must be reported separately from capability.