Grammar Control for Language Learning with Sequential Monte Carlo and Small Language Models
AuthorsKate McCurdy, Jacob Hoover Vigly, Thomas Hikaru Clark, Karolina Stańczak, Yahya Emara, Joshua B. Tenenbaum, Alexander K. Lew, Ryan Cotterell, Timothy J. O'Donnell
AffiliationsUniversity of Göttingen · CHI-FRO · MIT · ETH Zürich · Cornell University · Yale University · McGill University · Mila · Canada CIFAR AI Chair
A sampling-based method helps small language models produce dialogue that reliably practices specific grammar skills.
Key results
Grammar coverage after increasing the SMC budget to 8 particles, from a 58% baseline.
The 0.5B model is reported to outperform a model 140 times larger.
Llama 3.1 8B average coverage across target counts with 32 SMC particles.
Number of grammar skills targeted in the scaling experiment.
Share of English Grammar Profile skills for which GPT-5.4 generated viable potentials.
What the paper found
This paper uses sequential Monte Carlo, or SMC, to steer small language models toward grammar patterns chosen for English learners. Instead of retraining a model, it represents each grammar target as an inspectable Python potential: a fast prefix check guides partial text, while a final spaCy parse verifies the completed construction. With 8 particles, Qwen 2 0.5B rises from 58% to 98% target-grammar accuracy, exceeding the 96% result reported for Meta’s Llama 3.3 70B—a model 140 times larger. For dialogue tasks requiring as many as 6 grammar targets, Llama 3.1 8B reaches 78.1% average coverage with 32 particles, above the 68.5% hybrid fine-tuning-and-decoding baseline. The method also uses OpenAI’s GPT-5.4 and GPT-4o-mini to synthesize these grammar-checking programs: GPT-5.4 produced viable potentials for 99% of the 1222 skills in the English Grammar Profile, without supervised training data. The central trade-off is computation at inference time: more particles improve coverage, but slow generation and increase memory use.
Original abstract
Second language learners benefit from exposure to level-appropriate grammar in interactive language use. While prompted frontier language models can usually produce target grammar structures, less resource-intensive models that can be run on device often fail in this task. We use sequential Monte Carlo (SMC) to steer dialogue turn generation in small lan- language models toward target grammar constructions from an expert-developed pedagogical resource. With a minimal set of hand-coded potential functions, SMC achieves near-perfect coverage of target grammar, enabling small models to outperform state-of-the-art results from models 140 times larger. We also use code synthesis with frontier models to generate new potentials, showing that our method scales to hundreds of target grammar skills with no need for additional training or supervised data.
Read the original paperMore in Natural Language Processing
Browse all 29 papers →Contextual Embeddings Beyond the Gold Chunk
Sedigheh Eslami, Markus Krimmel, William Barber, Antoine Chaffin, Maximilian Schall, Jonas Golde, Louis Milliken, Ekaterina Glazkova, Denis Bykov, Bo Wang
A new retriever learns to surface both the chunks that answer a question and the evidence that supports those answers.
The uneven rise of AI-generated web content
Adam Visokay, Soumyakanti Pan, Molly Griston, Polina Iasakova, Mark Nepf, Denis Peskoff, Riya Sinha, Jikai Sun, Isabelle Thiara, Letao Wang, Yiwei Wu, Lauren Yan, Tyler H. McCormick
A study of over five million webpages finds that AI-written text is spreading unevenly, with the fastest growth on less-central pages.
AdaTutoRank: Learning to Rerank Document Sets via Adaptive Tutoring Optimization for RAG and Deep Research
Kailin Jiang, Lei Liu, Jian Xi, Yangqi Chen, Hui Xu, Hongwei Zhao, Bin Li, Yu Lu, Haibo Shi
AdaTutoRank teaches rerankers to assemble complementary evidence sets rather than merely picking individually relevant documents, improving RAG and deep-research retrieval with fewer calls.