Revitalizing Medical Time Series with Vision-Informed Retrieval: A Vision-Language Perspective

Medical time series (MedTS) analysis plays a critical role in supporting clinical decision-making and patient care. However, existing methods treat MedTS signals purely as numerical sequences, ignoring the rich information embedded in their visual waveform, which is essential for clinicians to diagnose physiological conditions. To bridge this gap, we introduce a novel Vision-Informed Retrieval (ViRe) framework that injects visual waveform prior knowledge into representations of raw MedTS sequences, thereby emphasizing morphology-relevant patterns and improving clinician-aligned reasoning. Specifically, a Vision Query is extracted using pre-trained vision-language models (VLMs) to obtain morphology-aware priors from waveform plots. To enrich the numerical representation with such morphology-aware information, we design a tailored attention-based cross-modal retrieval mechanism that aligns raw numerical features with these high-level vision priors through the Vision Query. ViRe demonstrates strong effectiveness, outperforming the state-of-the-art method by an average of 6.42% gain across six public MedTS benchmarks. Code is available at this Anonymous Repo.