Structure-Semantic Guided Closed-Loop Medical Anomaly Detection via Multi-Agent Collaboration

Unsupervised medical anomaly detection aims to identify images or regions that deviate from the learned normal distribution. Reconstruction-based methods are a dominant paradigm, generating a normal-looking reference for each test image and detecting anomalies via input--reconstruction discrepancy. However, most existing methods follow a one-shot reconstruction-and-comparison pipeline, making anomaly scoring vulnerable to reconstruction failures: abnormal regions may be preserved, while normal anatomical structures may be distorted. Moreover, they often lack coordinated control from structural and semantic normality, which can lead to anatomical inconsistency and semantic drift. We propose S2Agent, a structure- and semantic-guided multi-agent framework that reformulates medical anomaly detection as feedback-driven normality reconstruction. S2Agent decomposes the process into three collaborative agents. The Planner derives sample-specific matched-tree structural priors and normal-only semantic claims from normality knowledge. The Reconstructor generates a normality-oriented reference image under their joint guidance. The Detector verifies the reconstruction in matched-tree and semantic-claim spaces, returning feedback to refine structural weights and semantic constraints for the next round. Through this closed loop, S2Agent iteratively corrects structural deviation and semantic drift, suppresses abnormal or unsupported content, and improves the reliability of anomaly scoring. Experiments on three public medical benchmarks demonstrate the effectiveness of the proposed closed-loop guidance.