Beyond Unit-Circle Eigenvalues: Invariant Bases for Stable State Space Dynamics

Diagonal state space models process sequences efficiently but become unstable over long horizons: small spectral deviations compound into information decay or uncontrolled growth. For conservation-preserving or non-dissipative dynamics, constraining eigenvalues to the unit circle is a natural starting point and yields large short-horizon gains ($+3.31$ dB at step 10). We show that this is not enough: spectral constraints govern only the amplitude of each mode, not what each mode represents. The missing condition is alignment between the modal basis and the invariant structure of the underlying dynamics. Without this alignment, unit-circle constraints can preserve quantities unrelated to the task invariants and, on OOD scale-shift transfer, even degrade performance below unconstrained baselines. This establishes a broader principle: *stability in diagonal SSMs is a problem of representation, not just parameterization; eigenvalues determine how modes evolve, but only the basis determines what is preserved.* We instantiate this principle for isotropic pairwise systems through the Kronecker-Cayley Decomposition, which places dynamics in the graph Laplacian eigenbasis, yielding a diagonal SSM with exact layered conservation at $O(N^3+N^2m)$ cost instead of $O(N^3m^3)$, dominated by $O(N^2m)$ when $m \gg N$. Empirically, under our visual N-body protocol, a spectrally stabilized LRU baseline still accumulates $158\pm8$% momentum drift; controlled unit-modulus random and learned bases similarly fail ($120\pm18$% and $148\pm8$% drift); even PCA-derived bases, which empirically approximate the invariant direction, fall roughly three orders of magnitude short of the algebraic guarantee. Only the Laplacian basis yields drift below $10^{-4}$%, stable 1000-step rollouts (PSNR $21.19\pm1.27$ dB at step 999), and $+2.12$ dB OOD generalization on Charged particles.