Collocational bootstrapping: A hypothesis about the learning of subject-verb agreement in humans and neural networks

Claire Hobbs and R. Thomas McCoy, at Yale University, propose collocational bootstrapping as a mechanism for syntax learning in which learners infer subject-verb dependencies from word co-occurrence statistics rather than from explicit syntactic cues. They test this with 2-layer GPT-2-style decoder-only Transformers trained from scratch on 12,000 synthetic English-like sentences per condition, varying subject-verb pairing predictability with a truncated Zipf distribution parameterized by α from 0 to 3, plus an α→∞ no-variability case. Crucially, the training data were fully ambiguous between AGREE-SUBJECT and rival rules such as AGREE-RECENT. Performance on 1,000 minimal-pair test items per condition showed a sharp nonmonotonic effect: models failed when subject-verb pairings were too variable or too predictable, but generalized best at α≈1.4, where they achieved near-perfect accuracy even on unseen subject-verb pairs with intervening agreement attractors. The authors then parsed 4,739,189 adult-to-child utterances from CHILDES with spaCy, extracting 2,802,071 subject-verb pairs from the top 100 verbs, and found that the empirical co-occurrence distribution is also Zipfian, with best-fit α=1.43 overall and age-stratified values from 1.46 down to 1.23. The key novelty is the alignment between the model’s optimal variability regime and the statistical structure of child-directed English, suggesting that natural co-occurrence patterns may make abstract subject-verb agreement learnable through distributional cues alone.