Windowed-MTP: Removing the Full-Context Draft-KV Tax in Million-Token Agentic Serving

Agentic workloads make inference stateful. A long-horizon agent accumulates context across turns and tool calls until every decode step reads a million-token KV cache. Speculative decoding is the usual way to cut decode latency, and it assumes the draft is cheap. Frontier models increasingly ship a built-in Multi-Token-Prediction (MTP/NEXTN) draft head, and beside a hybrid-attention target at long context that head is not cheap. It attends over the full KV cache on each of its $\gamma$ forwards per decode step, while only 9-25% of the target's layers keep full attention, so a single draft layer does 50-75% as much context reading as the whole target. At 1M and $\gamma=6$, the draft alone takes 61-102% as long as the target's verify pass, and on hard, low-acceptance tasks deep native MTP becomes slower than no speculation at all. We give a per-decode-step cost model that predicts when this happens. We then apply a StreamingLLM-style window plus attention sink to the draft's attention only (Windowed-MTP). Verification still uses the full context, so the target distribution is preserved, and the change is training-free and drop-in. On three models from two hybrid families on one B200 in SGLang, it gives an acceptance-free 1.28-1.44× per-decode-step speedup at 1M, up from 1.17-1.27× at 261K. A compact ring buffer reclaims the draft KV pool the window no longer reads (7.7-11.1% of total KV), enough in the Qwen-35B batch sweep for one more concurrent 1M-token session on the same GPU. The gain also holds on 1M-token multi-turn dialogue (LongMemEval_M), at 1.08-1.84× over native.