Steering Embedded Language Flows with Formal Constraints

Embedded Language Flows (ELF) offer competitive text quality and sampling efficiency, but using them for generation under formal constraints requires explicit control over the decoded output. Their denoiser predicts contextual embeddings rather than token probabilities. Building on Diffinity, we develop training-free guidance that evaluates regular-expression acceptance through ELF’s native decoder and uses its gradient to steer the flow velocity. We further refine this guidance by averaging acceptance probabilities over Gaussian endpoint samples, with their variance estimated from the denoiser Jacobian trace. On 110 natural-language regular expressions spanning six categories, unguided ELF achieves 1.83% category-macro constraint satisfaction, and single-endpoint guidance reaches 87.25%. Our Jacobian-Covariance Monte Carlo Guidance (JC-MC), using four endpoint samples, raises satisfaction to 94.42%, with higher point estimates than the single-endpoint baseline in every category.