Keep the Receipts: Deletion Is the Only Catastrophic Action in LLM Context Curation
Medha Ravi
Abstract
Long-term memory systems for LLM agents increasingly handle conflicting information by deleting or hiding older facts once a newer statement is judged to supersede them. We test whether this retirement strategy is actually safe through controlled experiments on three memory benchmarks: LongMemEval ($n=500$), LoCoMo ($n=603$), and FactConsolidation ($n=600$). Holding retrieval and answering conditions fixed, we independently vary whether superseded turns are deleted or retained, whether retained turns receive CURRENT/OUTDATED labels, whether session dates are shown, and how much context is provided. Deleting superseded turns reduces answer accuracy by $15.2$ points overall and by $17.3$ points on multi-session questions. In $76%$ of deletion-specific failures, the gold answer appears in a deleted turn, showing that imperfect conflict resolution often removes answer-bearing evidence. A token-matched refill control recovers only $3.2$ points of the $14.4$-point deletion deficit, confirming that most of the harm comes from destroyed evidence rather than reduced context length. By contrast, correct labels add only $0.8$ points over an identical no-label condition, while even inverted labels have little aggregate effect because the model can override incorrect annotations using retained text and dates. Under controlled resolver corruption $\epsilon \in [0,0.5]$, retention-based policies remain stable, whereas deletion degrades monotonically. A quarantine policy—placing possibly superseded turns below a reference divider without removing them—matches or exceeds labeling on all three benchmarks and consistently dominates deletion. These results identify an asymmetric risk profile for context curation: deletion is irreversible and highly sensitive to resolver errors, while retention preserves recoverability and auditability. LLM memory systems should therefore retain dated evidence by default, use hedged quarantine when demotion is useful, and require exceptionally strong evidence before deleting information.
Chat is not available.
Successful Page Load