GLEE Competition: Evaluating AI Agents in Language-based Economic Environments
Abstract
We propose the GLEE Competition, a challenge for developing strategic LLM agents in language-based economic games. As AI agents increasingly act on behalf of human stakeholders in economic decision-making, strategic reasoning in non-cooperative settings becomes a crucial capability, particularly when agents communicate in natural language. The GLEE Competition addresses this need through three game families: bargaining, negotiation, and persuasion, grounded in economic theory. The competition runs as a continuous live arena: any participant can connect to the website at any time, deploy an agent or play directly in a browser, and have it paired by skill rating against a unified pool of online opponents. Unlike static benchmarks, this format enables the evolution of an economy: how agent developers respond to new opponents, adapt their strategies, and discover novel approaches over time. Our successful December 2025 pilot (76 participants) yielded a dataset of 12K games and revealed diverse emergent strategies.