When the Student Chooses Its Supervision: Adaptive State Exposure in Online Distillation

Continual post-training can couple learning to data collection: as a student changes, the states on which it requests supervision change with it. We isolate whether this endogenous supervision shift changes capability transfer. In a controlled sequential task, two teachers agree on common inputs but differ in one latent capability. For the primary comparison, we hold models, paired initializations, objective, and supervision budget fixed and vary only whether queries come from the initial or current student. Across 50 paired seeds, current-student queries raise held-out accuracy from 2.4% to 27.6%, while querying a capability-ablated teacher yields 0%. Continuous refresh is not intrinsically necessary: a mature frozen query source reaches 30.1% and is equivalent to online querying within a pre-specified 10-point margin. At fixed source maturity, greater exploration increases rare-state visitation, teacher disagreement on visited states, and transfer. Transfer nevertheless appears on only three of four held-out structures. Pre-specified depth-two queries raise accuracy on the missing local action to 84.0% but leave the required full sequence at 0% and degrade two supported structures. The student’s query policy thus changes the support of later supervision. In this setting, a mature frozen source matches online refresh, while neither local coverage nor continued interaction guarantees compositional transfer.