MARCH: Content-Routed State Anchors for Efficient Recurrent Memory

Long-context applications need both the direct recall of attention and the constant-memory decoding of recurrent models. We introduce Memory-Anchor Routing across Context History (MARCH), a recurrent architecture that preserves periodic snapshots of a continuously updated state and retrieves them by content. Each snapshot is paired with a compact routing key; a token-dependent query selects causally available anchors, reads their weighted state mixture, and fuses it with the current recurrent readout. MARCH therefore expands temporal memory capacity without changing the underlying Gated DeltaNet recurrence or requiring token-level key-value storage. In 50B-token pretraining experiments, MARCH improves over strong recurrent baselines on commonsense reasoning, LongBench, and in-context retrieval, while retaining a favorable efficiency trade-off through sparse routing and controllable checkpoint density.