NTH

Grammar Control for Language Learning with Sequential Monte Carlo and Small Language Models

AuthorsKate McCurdy, Jacob Hoover Vigly, Thomas Hikaru Clark, Karolina Stańczak, Yahya Emara, Joshua B. Tenenbaum, Alexander K. Lew, Ryan Cotterell, Timothy J. O'Donnell

AffiliationsUniversity of Göttingen · CHI-FRO · MIT · ETH Zürich · Cornell University · Yale University · McGill University · Mila · Canada CIFAR AI Chair

November 12, 2026 2 min read
Watch on YouTube
The one-line take

A sampling-based method helps small language models produce dialogue that reliably practices specific grammar skills.

Key results

98%
Qwen 2 0.5B accuracy

Grammar coverage after increasing the SMC budget to 8 particles, from a 58% baseline.

140
Model size advantage

The 0.5B model is reported to outperform a model 140 times larger.

78.1%
Six-target grammar coverage

Llama 3.1 8B average coverage across target counts with 32 SMC particles.

1222
English Grammar Profile skills

Number of grammar skills targeted in the scaling experiment.

99%
Viable generated potentials

Share of English Grammar Profile skills for which GPT-5.4 generated viable potentials.

What the paper found

This paper uses sequential Monte Carlo, or SMC, to steer small language models toward grammar patterns chosen for English learners. Instead of retraining a model, it represents each grammar target as an inspectable Python potential: a fast prefix check guides partial text, while a final spaCy parse verifies the completed construction. With 8 particles, Qwen 2 0.5B rises from 58% to 98% target-grammar accuracy, exceeding the 96% result reported for Meta’s Llama 3.3 70B—a model 140 times larger. For dialogue tasks requiring as many as 6 grammar targets, Llama 3.1 8B reaches 78.1% average coverage with 32 particles, above the 68.5% hybrid fine-tuning-and-decoding baseline. The method also uses OpenAI’s GPT-5.4 and GPT-4o-mini to synthesize these grammar-checking programs: GPT-5.4 produced viable potentials for 99% of the 1222 skills in the English Grammar Profile, without supervised training data. The central trade-off is computation at inference time: more particles improve coverage, but slow generation and increase memory use.

Original abstract

Second language learners benefit from exposure to level-appropriate grammar in interactive language use. While prompted frontier language models can usually produce target grammar structures, less resource-intensive models that can be run on device often fail in this task. We use sequential Monte Carlo (SMC) to steer dialogue turn generation in small lan- language models toward target grammar constructions from an expert-developed pedagogical resource. With a minimal set of hand-coded potential functions, SMC achieves near-perfect coverage of target grammar, enabling small models to outperform state-of-the-art results from models 140 times larger. We also use code synthesis with frontier models to generate new potentials, showing that our method scales to hundreds of target grammar skills with no need for additional training or supervised data.

Read the original paper

More in Natural Language Processing

Browse all 29 papers →
01Nlp

Contextual Embeddings Beyond the Gold Chunk

Sedigheh Eslami, Markus Krimmel, William Barber, Antoine Chaffin, Maximilian Schall, Jonas Golde, Louis Milliken, Ekaterina Glazkova, Denis Bykov, Bo Wang

A new retriever learns to surface both the chunks that answer a question and the evidence that supports those answers.

Read analysis
02Nlp

The uneven rise of AI-generated web content

Adam Visokay, Soumyakanti Pan, Molly Griston, Polina Iasakova, Mark Nepf, Denis Peskoff, Riya Sinha, Jikai Sun, Isabelle Thiara, Letao Wang, Yiwei Wu, Lauren Yan, Tyler H. McCormick

A study of over five million webpages finds that AI-written text is spreading unevenly, with the fastest growth on less-central pages.

Read analysis