The Piggyback Hypothesis of Generalization: Explaining and Mitigating Emergent Misalignment

The mechanisms behind LLMs' broad generalization beyond training examples are poorly understood. Emergent misalignment (EM) offers a striking case study: finetuning on narrow tasks induce broad misalignment to semantically-unrelated test domains. In this work, we propose the Piggyback Hypothesis: the chat-template prefix, shared across all user queries, can piggyback the finetuned behavior onto out-of-domain queries. We validate this hypothesis by showing that subtle perturbations to the prefix, or simply patching the prefix representations with those from the unfinetuned model, can restore alignment without changing the user query. Building on this finding, we propose Token-Regularized Finetuning (TReFT), which directly regularizes prefix representations during training to mitigate piggybacking. Across Llama-3.1, Qwen-2.5, and GPT-OSS models and multiple EM-inducing datasets, TReFT reduces EM while preserving in-domain learning. On Llama-3.1-8B finetuned on the legal domain, TReFT achieves 33.5\% more EM reduction than data interleaving with a retain set of aligned examples. We further show that TReFT extends to other narrow-finetuning settings, including abstention, tool use, and refusal (off-topic generalization is reduced by 54.3\% on average), which further supports the Piggyback Hypothesis. Broadly, our work highlights that LLMs may learn and generalize in unintended ways and suggests a path toward more controllable finetuning. It also calls for further study of how shared input features can piggyback model behavior across domains.