It Cancels: O(r²) Cholesky Updates for Whitened Operators

Many sequential Bayesian methods maintain a whitened operator $A_w = L^{-1}ML^{-\top}$, where $L$ is the Cholesky factor of a kernel, precision, or curvature matrix. When the underlying matrix changes by a rank-$1$ perturbation, existing techniques propagate only one-sided quantities $L^{-1}B$; refreshing the sandwiched operator $A_w$ requires a full $O(r^3)$ re-factorization that dominates iteration cost, consuming up to 91% of per-step time in natural gradient methods. We show that this cost is unnecessary. For pre-whitened inputs, intermediate factors of $L$ cancel exactly from both sides of the augmented Givens rotations, and the updated $(L^+)^{-1}M^+(L^+)^{-\top}$ can be recovered in $O(r^2)$ time by applying the rotations double-sidedly to a zero-padded matrix, followed by a rank-1 correction. We provide a self-contained derivation, stability bounds, and implementations as a C++ extension, MLX kernel, and Triton GPU kernel. On streaming sparse GP regression, online Bayesian optimization on protein fitness landscapes, and natural gradient preconditioning, the method matches full re-factorization in accuracy with up to $5.4\times$ speedups. At $r = 1024$, training time drops from 1075s to 198s, making larger curvature charts affordable and producing strictly better test MSE.