From Autoregression to Block Diffusion: Adapting Language Models for Efficient Parallel Decoding

Small Language Models (SLMs) are now capable enough for real workloads on consumer and embedded hardware, where decoding cost increasingly dominates. Autoregressive generation requires one forward pass per emitted token, making long agentic and multi-turn workloads particularly expensive. Diffusion language models tackle this issue by replacing token-by-token causal decoding with bidirectional parallel refinement, reducing the number of forward passes needed for long outputs. However, they still suffer from a quality gap relative to autoregressive models, especially at the small scale. In this paper, we propose a compact recipe for adapting a pre-trained autoregressive model into a block-causal diffusion model, together with a fused-kernel serving stack for the result. We apply our methodology to LFM2.5-350M as a case study. At 8 denoising steps per block (4× faster than autoregressive decoding on an H100 at batch size 1), the model matches or nearly matches its autoregressive source on most benchmarks.