Constructing an Obligation-Aware Evaluation Corpus for Enterprise Agent Memory

Long-term memory allows enterprise agents to preserve context, create derived records, and continue work across sessions. Most memory evaluations focus on recall or forgetting and provide little evidence about whether a system can meet record-specific obligations governing access, retention, correction, and deletion. We present a method for constructing obligation-aware memory evaluations from concrete record actions across six financial applications. Each scenario links a record’s provenance and lifecycle events to an expected outcome grounded in cited public sources under explicit assumptions. The resulting corpus contains 52 scenarios expressed through nine composable memory primitives. A three-stage diagnostic locates failures in the agent’s decision, the system’s capability, or the durability of execution. We assess whether the primitives cover and compose across the corpus, whether labels trace back to their sources, and whether the diagnostic separates distinct failure types. This takes memory evaluation beyond remembering and forgetting to the governance of records over time.