Test-Time Personalization: A Diagnostic Framework and Probabilistic Fix for Scaling Failures

Existing approaches to LLM personalization focus on constructing better personalized models or inputs, while treating inference as a single-shot process. In this work, we study \textbf{Test-Time Personalization (TTP)} along an unexplored axis: scaling inference-time computation by sampling $N$ candidates from a personalized policy model and selecting the best with a personalized reward model. However, standard reward models fail to realize this potential. To diagnose why, we derive a unified scaling law that decomposes any reward model's Best-of-$N$ curve into four measurable quantities and reveals two failure modes, \emph{user-level collapse} (near-constant prediction for some users) and \emph{query-level reward hacking} (negative correlation with true quality for some queries). Guided by this law, we propose a probabilistic personalized reward model whose learned variance effectively mitigates both failure modes. Experiments confirm both elements of our framework: TTP delivers consistent scaling across multiple policy models and personalized text generation tasks, and our scaling law closely matches observed scaling curves across reward-model variants.