SignRot: LLM Quantization with Massive Outlier-Aware Sign-Adjusted Rotation

While quantization compresses LLMs and accelerates inference, outlier activations impede efficient low-bit representation. In particular, massive outliers cause substantial performance degradation. Although rotation-based methods that rely on randomized Hadamard transforms for online inference aim to mitigate this, they consistently produce fragmented distributions with empty bins when encountering massive outliers. This fragmentation severely undermines uniform quantization performance. Moreover, when massive outliers occur in down-projection activations, the rotation must be applied at inference time. This constraint prevents training-based approaches such as SpinQuant from learning the online rotation at this position, as a learned dense matrix would forgo the Fast Walsh--Hadamard Transform and require a full matrix multiplication during inference. To address this, we propose SignRot (Sign-adjusted Rotation), which identifies massive outlier channels, and constructs a sign adjustment vector that unifies the signs of the corresponding Hadamard row, collapsing the bimodal distribution into a unimodal profile amenable to quantization. As a lightweight, drop-in improvement compatible with any Hadamard-based quantization method, SignRot requires only a single calibration forward pass and introduces no additional inference overhead. Applying SignRot to QuaRot reduces the Wikitext2 perplexity to 5.88 and increases the average zero-shot reasoning accuracy to 72.73, surpassing SpinQuant on the LLaMA3-70B model.