Efficient Forecasting of Task Failures in LLM Agents through Adaptive Fault Injection

LLM agents often execute long-horizon tasks where sequential tool calls and reasoning steps compound into a final outcome. When an agent errs mid-execution, the critical question is not whether an error occurred, but whether recovery under the current policy is still possible. Existing failure analysis answers this retrospectively, making it too late for runtime intervention. We introduce Task Failure Forecasting: predicting from a partial execution trace whether continued execution under the current agent policy is likely to fail. This is hard: human experts achieve only 54\% accuracy at distinguishing recoverable from failure-inducing errors, and frontier models such as Claude-4.5 Sonnet reach 59.87\% weighted F1. The core obstacle is that failed traces mix recoverable mistakes with fatal ones, obscuring which steps drive downstream failure. We address this with adaptive fault injection: injects targeted perturbations into successful traces, uses bandit-based prioritization of high failure yield error types and rollout-based verification providing policy-conditioned step-level supervision. A lightweight Qwen-3-8B model trained on this synthetic data achieves 78.01\% weighted F1 on human-annotated benchmarks, outperforming Claude-4.5 Sonnet by 14 weighted-F1 points at 100$\times$ lower forecasting cost. When deployed as a runtime monitor for selective inference-time intervention, the forecaster improves agent success rates across HotpotQA, MuSiQue, GAIA, AssistantBench, SWE-Bench, MBPP and EnterpriseBench, yielding absolute gains in the range of 2–16 \% points across held-out and unseen tasks.