Anchoring Adversarial Trajectories to Data Manifolds: A Bilevel Transfer Optimization Framework

A key bottleneck in adversarial transfer is a *trajectory-level geometric disconnect*: ambient gradients often drift away from the intrinsic data manifold, causing surrogate-specific overfitting. To rectify this, we propose *Manifold Anchored Bilevel Transfer (MABT)*, a unified framework that anchors adversarial trajectories to the shared semantic subspace. MABT introduces a relaxed manifold-anchoring operator as a semantic rectifier to suppress off-manifold noise. With this constraint, we cast transfer attack generation as a distributional bilevel optimization problem that learns a geometry-aligned initialization by minimizing expected transfer risk under a surrogate uncertainty distribution. We further develop a Hessian-free solver with linear-time complexity to handle the resulting hierarchy. Experiments demonstrate that MABT consistently boosts the transferability of **10** baselines across diverse attack scenarios and defense mechanisms (e.g., **63.31\%** average ASR $\uparrow$ across **28** attacker combinations).