Democratic Perspectives and Corporate Captures of Crowdsourced Evaluations

This provocation is a response to a growing trend in large language model (LLM) evaluation: AI companies and researchers are increasingly promoting evaluations that rely on crowdsourced labor and framing this shift as a "democratization" of LLM development. We argue that this version of democracy prioritizes participation in a format that is ingestible by AI companies over democratic principles like deliberation, discourse, dissent, and disobedience that are widely recognized as enriching to democratic processes. We also posit that contribution is capture, referring to the fact that a user contributing to the preference extraction systems sacrifices their time, labor, attention, and data to ultimately produce additional value for AI companies. We believe it is critical that evaluations incorporate new modes of participation to expand the scope and impact of crowdworker input, while recognizing that this legitimization of crowdsourced evaluation further exacerbates the phenomenon of contribution as capture. We conclude that the seeming impossibility of addressing both critiques is neither necessary nor universal, for power dynamics of language models can be fundamentally restructured and a freeing, democratic AI evaluation can be realized.