Self-Normalized Martingales under Weak Moments: Anytime-Valid Confidence Bounds

We derive anytime-valid confidence bounds for matrix-valued self-normalized martingales with predictable covariates and vector noise under a bounded conditional moment of any order $1+\eta>1$. Pseudo-maximization elegantly yields confidence bounds under light-tailed noise, but extending it to weak moment assumptions with a covariate-based normalizer remains challenging. Our proof combines a regularized form of a classical elementary exact decomposition (Lai and Wei, 1982; Lai 1986). with martingale moment inequalities, retaining the decomposition's predictable negative offset. The bounds control the squared Frobenius norm without requiring symmetry or bounded covariates. Under bounded noise, our proof technique yields an improved Bernstein bound, relaxing the design requirements of a PAC-Bayes approach and sharpening the confidence dependence of a Freedman-based approach. We apply these results to anytime regret minimization in stochastic linear bandits with changing action sets using ordinary ridge least squares.