Habitats: A Framework for Open-Ended Evolution of Agent Objectives

Designing what an agent should optimize can be as difficult as optimizing the agent itself. Existing language-model-based reward-generation methods typically refine objectives for predefined tasks and couple the decision of *what* to explore with the synthesis of *how* to implement it, potentially narrowing the resulting behavioral repertoire. We introduce **Habitats**, a framework for open-ended evolution of executable agent objectives. Habitats maintains a population of reward-policy pairs and evaluates every policy under every population reward, producing a cross-fitness matrix that exposes self-mastery, transfer, and functional diversity. A population-level Planner uses this information to select parent rewards and issue targeted mutation directives, while a separate Mutator translates those directives into validated reward programs. We evaluate Habitats in a discrete Card Sandbox and three continuous-control environments (Cart-Pole, Hopper, and Cheetah) against Independent Generation and Direct Mutation baselines. Across four environments and three random seeds, Habitats achieves the highest macro-averaged final Specialist Quality and Behavioral Diversity, with final diversity reaching $1.8\times$ that of Direct Mutation and $2.5\times$ that of Independent Generation. Finally, across ten downstream objectives unseen during evolution, frozen Habitats policy libraries reach mean parity with task-specific policies trained from scratch using half as many downstream-training episodes. These results suggest that separating population-level planning from reward synthesis enables the discovery of diverse, learnable, and reusable behaviors.