Fuzzy Glade: Evidence-Grounded, Agent-Guided Analysis of Agent Trajectories at Scale
Abstract
As agents get more capable, model training requires richer behavioral signal than what static evaluation metrics like pass rate offer. Agent trajectories are a gold mine for such insights, but their size, volume, and granularity make them difficult to analyze for high-quality insights. We introduce Fuzzy Glade (FG)—a comprehensive framework and application for performing agent-guided analysis of agent trajectories at scale in a reproducible, evidence-annotated, and easy to review manner. We also present a case study from a real use of FG in the model training of our production model—detecting and profiling reward-hacking behavior in evaluation and reinforcement-learning rollouts. We release the reward-hacking dataset sourced from this use case alongside human annotated ground truth as an evaluation of the tool.