RGB-SIM: Reference-Guided Behavioral Steering for Behaviorally Representative User Simulation

In this paper, we study the difficulty of LLM-based user simulators in reliably adhering to specified behaviors, which causes simulated conversations to diverge from human behavioral patterns. We first develop a feature-extraction framework that quantifies this divergence across seven behavioral dimensions grounded in prior work, revealing substantial distributional mismatch and highlighting standard LLM-based simulators’ limited ability to realize specified behaviors. To reduce this gap, we propose RGB-SIM, a training-free framework that estimates behavioral target vectors from input specifications using a small reference set and dynamically translates them into turn-level steering instructions for more reliable behavioral realization. Across three model families and two task-oriented dialogue benchmarks, RGB-SIM improves adherence to specified behaviors, producing more behaviorally representative conversations and reducing KL divergence by up to 6.98 absolute points relative to vanilla simulation.