Can LLMs Uncover Hidden Physical Laws through Free Exploration?
AuthorsZhen Yang, Wen Wang, Xander Xu, Ying-Cong Chen
AffiliationsHKUST(GZ) · ZJU · Alibaba Group · HKUST
Researchers let LLMs freely explore simulated worlds with altered physics to see whether they could uncover the hidden laws.
Key results
One ordinary-physics control and 14 scenes with altered laws.
Only model to pass both law-correctness and evidence-support checks under free exploration.
The remaining 11 evaluated models made no discoveries.
Fraction of runs identifying the altered law sufficiently completely.
Fraction of runs whose findings were supported by recorded experiments.
Rate when GPT-6-Astra received a scene-specific clue and a discovery task.
What the paper found
Can LLMs uncover physical laws without being told what to investigate? LongChang tests this in MuJoCo simulations where explorers manipulate objects, build measuring tools, and choose their own questions. Its 15-scene base set contains a normal-physics control and 14 worlds, each with one hidden alteration, such as mass-dependent falling or a force that pauses after an object is pushed. Separate LLM agents review tools and findings, then use exploration records to generate, refine, and combine scenes for further discovery. Across 12 models—including GPT-6-Astra, Claude-Opus-5, Gemini-3.7-Flash, DeepSeek-Flash, Qwen3-Max, and Grok-4.7—GPT-6-Astra was the only model to achieve a nonzero discovery rate: 16.7%, while the other 11 scored 0%. Its law-correctness rate was 19.0%, despite 97.6% evidence support, showing that experiments often supported observations without establishing the full law and its conditions. Providing a scene-specific clue raised GPT-6-Astra’s discovery rate to 28.6%, suggesting that noticing an anomaly and turning it into a complete, tested explanation remain distinct challenges.
Original abstract
Scientific discovery often begins with an unexpected observation that prompts a new research question. In a world with altered physical laws, can large language models (LLMs) notice unexpected behavior, such as heavier objects falling more slowly, and discover the law through experiments without being told what to investigate? To evaluate this ability, we introduce LongChang, an automated framework that creates simulated worlds with altered physics, where one or more laws differ from ordinary physics. The evaluated LLMs act as explorers, manipulating objects and building tools to investigate their own questions without research tasks or hints about these laws. Separate LLM agents generate and review scenes, review tool designs, and evaluate reports for correctness and experimental support using the altered physical laws and exploration records. These records guide scene evolution for further exploration, allowing the full cycle to run without human intervention. Among 12 LLMs, GPT-6-Astra achieves a discovery rate of 16.7%, while the other 11 score 0%. The exploration records reveal uninvestigated anomalies and only partial identification of the altered physical laws.
Read the original paperMore in AI for Science
Browse all 52 papers →Can Test-Time Memory Evolution Outperform Test-Time Training for Scientific Discovery?
Yepeng Liu, Shengmai Chen, Xuandong Zhao, Chengzhi Liu, Dawn Song, Xin Eric Wang, Yuheng Bu
The paper proposes improving AI-driven mathematical discovery by evolving its memory instead of retraining its weights at test time.
LLM2Cert: LLM Synthesis of Certificates with SMT/Lean Verification
Zhipeng Cao, Peiyang Song, Nikolaus Vertovec, Weiru Cui, Zhihao Yang, Mingkang Liu, Luxia Lin, Bai Xue, Alessandro Abate
LLM2Cert uses language models and formal tools to propose and verify mathematical safety and stability certificates for dynamical systems.
Automating ML for Science: Can Frontier Agents Climb Scientific Hills in the Wild?
Ming Zhong, Stacy Li, Nicholas Carlini, Matthew Jagielski
Agent4Sci tests how far frontier AI agents can improve real scientific ML models—and finds they need more than engineering tricks to make research-level progress.