Grassmannian Geodesic Steering: Rank-Preserving Subspace Control for Inference-Time Alignment of Language Models

Inference-time activation steering provides a lightweight mechanism for aligning language model behavior without retraining, yet existing additive interventions often distort activation geometry, alter representation norms, and induce effective-rank collapse, thereby degrading open-ended generation. Recent norm-preserving rotation-based methods mitigate this pathology, but their reliance on one- or two-dimensional concept axes is insufficient for behaviors whose representations are intrinsically multi-faceted and low-rank. We propose Grassmannian Geodesic Steering (GGS), a rank-preserving framework that lifts activation control from vector directions to $k$-dimensional subspaces on the Grassmann manifold. We develop a theoretical account showing that additive steering provably collapses effective rank under strong intervention, that low-dimensional rotations cannot capture distributed behavioral concepts, and that orthogonal transformations along Grassmannian geodesics exactly preserve the singular-value spectrum. Guided by these results, GGS estimates concept subspaces from contrastive difference covariance, steers hidden states via a closed-form Grassmannian geodesic realized as an orthogonal operator, and adaptively modulates intervention strength using a matrix Bingham gate over subspace-valued activations. The resulting method introduces less than $1.5%$ wall-clock overhead per decoding step. Across LLaMA-3.1-8B and Qwen-2.5-7B on six reasoning benchmarks and an AdvBench safety evaluation, GGS consistently improves alignment accuracy while preserving generation quality and refusal behavior, reducing effective-rank degradation by two orders of magnitude relative to additive baselines, and surpasses the current state of the art among training-free inference-time steering methods.