A Method-Class Divide in Sub-4-Bit Quantization:\\Iterative vs.\ Single-Pass Sensitivity to Data Composition
Yehong Jiang ⋅ Yen-Kuang Chen ⋅ Xinmin Tian
Abstract
Block-wise post-training quantization (PTQ) compresses LLMs by minimizing per-block reconstruction error on a small calibration set. We show that the \emph{composition} of this set (200 samples) is a targeted lever at extreme bit widths, acting only on iterative methods. In a controlled study spanning six data-dependent PTQ methods plus a data-free baseline, six models (1.7B--8B) across three families plus 70B verification, three bit widths, and a 10-task evaluation with four-seed replication, we find a sharp \textbf{near-cliff method-class divide}. In each method's pre-cliff regime (where the model retains predictive function), iterative methods (SignRound~V2, OmniQuant, AQLM) gain 0.74--1.93pp on the 10-task average from switching Pile to a fragility-guided dataset (FragCal-DC), while single-pass analytical methods (GPTQ, AWQ) show near-zero response ($|\Delta|{\le}0.30$pp). Both classes are evaluated on functional Pile baselines (0.42--0.61, well above chance); above the cliff, every method becomes data-composition-insensitive. A reconstruction-loss paradox (the method that most reduces its proxy loss gains nothing downstream, while the method that barely moves the proxy gains the most) sharpens the mechanism; fixed-reference held-out evaluation confirms the proxy--downstream disconnect. A wrong-answer contamination ablation retains 87--91\% of the improvement and a data-dependent single-pass control (RTN-opt) shows zero response, isolating the iterative optimization loop. A weight-space probe shows SignRound-V2 preserves higher per-group scale heterogeneity than GPTQ on 9 of 9 (model, width) cells; a scale-flatten knockout shows this heterogeneity is necessary for iterative fidelity. We release FragCal-DC, the discriminative-task instance used here, as a drop-in JSON for the evaluated setting.
Chat is not available.
Successful Page Load