NTH

Curriculum unlearning in humans and neural networks

AuthorsJirko Rubruck, Alice Zhang, Andrew Saxe, Christopher Summerfield

AffiliationsDepartment of Experimental Psychology, University of Oxford, Oxford, UK · Gatsby Unit & Sainsbury Wellcome Centre, University College London, London, UK*These authors contributed equally to this work

November 7, 2026 2 min read
Watch on YouTube
The one-line take

Simple-to-complex curricula may work by helping humans and neural networks unlearn overly simple rules and build richer representations.

Key results

65.3%
Human rule-plus-exception strategy at test

Simple-to-complex curriculum, compared with 44.4% without a curriculum.

1.09
Participation ratio after 1-D pretraining

Network representation was strongly compressed before the rule-plus-exception task.

2.93
Participation ratio after XOR pretraining

Network representation was more dimensional after XOR pretraining.

0.051
Exception-accuracy gain after 1-D pretraining

Simple-to-complex curriculum benefit in networks.

0.018
Exception-accuracy gain after XOR pretraining

Simple-to-complex curriculum benefit in networks.

What the paper found

In a rule-plus-exception categorization task, where six of eight stimuli follow a simple rule and two violate it, humans and neural networks first favored the simple rule and often struggled to learn the exceptions. The study’s central finding is that presenting rule-consistent examples before gradually introducing exceptions helps learners escape this overly compressed solution—a process the researchers call curriculum unlearning. In the first experiment, 65.3% of participants trained with a simple-to-complex curriculum used the correct rule-plus-exception strategy at test, compared with 44.4% without a curriculum. A two-layer network with 32 ReLU hidden units showed the same pattern: its hidden representation’s participation ratio rose from 2.27 without a curriculum to 2.37 with one, indicating greater task-relevant dimensionality. A second experiment with 1,027 participants and matched networks tested whether prior learning changes the benefit: one-dimensional pretraining compressed network representations to a participation ratio of 1.09, while XOR pretraining produced 2.93. The curriculum’s exception-accuracy gain was 0.051 after one-dimensional pretraining versus 0.018 after XOR pretraining. Together, the behavioral and representational results suggest that curricula help most when learners have already compressed away dimensions needed to master exceptions.

Original abstract

During learning, humans often benefit from curricula that teach simple rules before exceptions. However, we lack a computational explanation for this phenomenon. Here, using a canonical behavioural task and modelling with neural networks, we develop and test a theory for why simple-to-complex curricula succeed. We show that during learning, both humans and neural networks show simplicity biases, a tendency to learn simple rules that are underfit to the data. In neural networks this bias is accompanied by compression of neural representations. Simple-to-complex curricula help both types of learners overcome this compression, developing representations (and policy) with appropriate dimensionality for the task. We test and confirm a further prediction from this theory: learners that have been pre-exposed to simple tasks and with more compressed initial representations benefit more from curricula. These findings suggest that curricula help with ``unlearning'' of simple rules.

Read the original paper

More in Neural Networks

Browse all 28 papers →
01Neural Network

Do vision models know that three is three?

Ankit Sonthalia, Arnas Uselis, Seong Joon Oh

Vision models often learn a concept change clearly within one visual setting, but may represent that same change differently in another.

Read analysis
02Neural Network

Meta Models: Models that Model Learning

Peihang Li, Ziming Liu

Instead of running a model-training experiment first, the authors use prior runs to predict how the next one will learn.

Read analysis
03Neural Network

Continuous Memory Machines

Ciaran Regan, Kai Arulkumaran, Luke Darlow, Stefania Druga, Sebastian Risi, Llion Jones

Continuous Memory Machines give recurrent networks separate short- and long-term memories that can read from and reorganize one another.

Read analysis