LLM trained solely on K, 5 curriculum hits hard knowledge ceiling at fifth-grade level

The experiment constrains the training corpus to an 88B-token dataset filtered to U.S. elementary-school curriculum standards (K, 5). This creates a hard boundary: concepts, facts, and vocabulary taught above fifth grade are explicitly excluded. Models trained on this corpus, LittleLearner at 0.6B, 1.3B, and 5B scales, are matched with unfiltered controls for direct comparison. The setup isolates the role of pretraining data in shaping model capability, removing ambiguity about whether new skills are learned or merely elicited from broader exposure.

Scaling model size improves performance within the K, 5 scope and extends modestly to problems along the same learning trajectory. However, it yields little to no improvement on tasks requiring knowledge beyond the fifth-grade curriculum. Post-training through GRPO boosts in-scope capabilities but fails to recover out-of-scope performance, even when trained on beyond-K, 5 data. In-context learning similarly does not unlock new reasoning capabilities for beyond-K, 5 tasks. These findings suggest that the pretraining filter sets an effective ceiling on model capability, and standard interventions do not meaningfully push beyond it.

The controlled exposure enables clean experiments on knowledge acquisition. For example, introducing new concepts like negative numbers allows measurement of sample efficiency, retention, and interference. The boundary also facilitates comparisons between machine and human learners, such as whether models and children require similar exposure to learn fractions or make similar errors on word problems. This makes LittleLearner a tractable proxy for studying reward-driven discovery, continual learning, and educational science in a way that is difficult with unconstrained models.

For engineers, the experiment highlights the limits of scaling and post-training when pretraining data is restricted. It suggests that expanding model capability may require deliberate curation of training data rather than relying on interventions applied after the fact. The findings also underscore the importance of dataset design in shaping model behavior, particularly for applications where knowledge boundaries must be predictable or interpretable. The sandbox nature of LittleLearner offers a tool for testing hypotheses about learning dynamics without the noise of broader, uncontrolled pretraining.