When Similarity Is Not Redundancy: Temporal Compression Under Continuation
Abstract
At a declared memory boundary, a temporal summary can preserve the correct current answer while discarding the information needed to update it. We study this gap using paired histories that share endpoints, state multisets, duration, and current labels, but require opposite answers after the same continuation. Similarity- based reference deletions and mergers collapse these histories even when all features are mutually close. An exact collision of the complete retained state gives a decoder-independent obstruction. We connect the construction to origi- nal token-compression operators acting on frozen ImageNet-pretrained features: fixed-threshold deletion produces collisions on six paired synthetic histories, in- cluding after the observation grid is refined. Matched-token sampling, last-anchor, geometry, and order-blind scalar controls localize the effect; tested budgeted merg- ing settings do not exhibit it. A task-specific accumulator preserves the missing distinction. These results motivate continuation-based audits of reusable mem- ory, not a claim of failure by complete video-language models or a new general sufficient-state theory.