Development Builds Expectations: How Structured Experience Shapes Future Learning in Transformers
Abstract
Prior knowledge changes how new experience is learned, but it is difficult to isolate this developmental interaction in biological learners. We construct a controlled synthetic world in which small Transformers develop probabilistic knowledge schemas and later encounter matched facts that are consistent with, inconsistent with, or neutral to those schemas. Across 12 preregistered seeds, schema congruence increasingly shaped acquisition over developmental training: the mature congruence advantage was 0.897 [0.762, 1.063], and its increase from early development was 0.519 [0.378, 0.699]. Post-hoc decomposition showed that this advantage arose largely from developed expectations; after subtracting pre-exposure beliefs, schema violations instead induced larger corrective updates. Inconsistent learning also produced substantially greater interference with established knowledge and reorganization of its causal-support structure, effects not explained by generic learning difficulty. A separate causal manipulation of schema regularity increased expectations and corrective updating but not mature interference, with secondary analyses indicating stronger sensitivity early in development. These results suggest that developmental experience progressively transforms future learning from relatively expectation-free acquisition into prior-guided plasticity, while revealing limits to simple circuit-reuse accounts of schema assimilation.