Diversity and Stability in LLM Agent Populations: The Social--Modal Variance Trade-off
Abstract
I show that variance in judgments caused by probabilistically generated evidence can manifest in two different places: as differences between agents within a population, \textit{social variance}, or as differences between possible aggregate states of the population, \textit{modal variance}. This matters for LLM-based social simulation because the information available to agents is itself part of the system design: agents may receive private observations, overlapping retrieved documents, or common message histories. A simple network model shows how greater overlap between agents' evidence pools decreases social variance while increasing modal variance. The same population can therefore look highly consensual within a run while being sensitive to which evidence happened to be shared in that run. Conversely, less-correlated evidence can generate more disagreement while stabilizing aggregate outcomes across runs. The dynamic is upstream of particular updating or reasoning procedures and is most consequential for problems that are challenging yet tractable. I draw out implications for interpreting consensus, validating synthetic populations, and designing multi-agent systems, where the dependence between agents' information pools is itself a design choice.