Transfer without target: hidden failure in cross-cohort perturbation prediction

Controlled human infection models (CHIMs) are valuable for evaluating vaccine responses, but their restriction to adults leaves post-vaccination infection responses unobserved in important target populations such as infants. We study this as a small- sample, cross-cohort transfer problem in vaccine immunology: predicting an infant post-infection-challenge transcriptomic state that cannot ethically be measured, using adult controlled-human-infection data and an infant vaccine-only cohort. On a synthetic benchmark with known ground truth, standard approaches - including di- rect expression-space regression, class-imbalance oversamplers (SMOTE, LoRAS), and off-the-shelf domain adaptation tools (non-parametric optimal-transport (OT) transport, CORAL), fail to recover the target state reliably. We investigate failure using a synthetic dataset with a known truth value, which is unavailable in the real problem setting. Reframing the problem in shift space resolves the first failure mode: it is the only tested approach that stays anchored in the correct cohort’s feature space and retains directional perturbation signal. Following on from this, the same synthetic ground truth exposes a second failure mode inside the shift- space method itself: a standard ridge-regression step consistently under-recovers true perturbation magnitude, a failure mode invisible to the validation diagnostics available on the real data. This reveals a distinction between learning the direction of a transferable perturbation and recovering its magnitude: a model can appear well fitted under conventional error metrics while substantially attenuating the biological response of interest. We partially mitigate this failure mode using partial least squares regression.