Time-Aware Contrastive Transformer for Longitudinal Patient Representation Learning

Learning high-quality longitudinal patient representations from irregular electronic health records (EHRs) is essential for understanding disease progression of time-evolving diseases, such as cancer, in a heterogeneous population. Longitudinal patient representation learning methods often rely on external labels for downstream tasks or do not model the temporal dynamics between medical events explicitly, which makes the learned disease trajectories clinically less relevant. In this work, we propose the Time-Aware-Contrastive-Transformer (TACT), a transformer-based model that integrates explicit temporal modeling with a fully self-supervised contrastive learning framework. We introduce a sampling-based data augmentation workflow that leverages the internal structure of clinical events together with the hierarchical taxonomies of diagnoses and medications to enrich representation learning. Evaluated on a real-world five-year timeseries of 77,628 cancer patients, TACT demonstrates robust performance across patient representation and event embedding metrics and outperforms two time-aware transformer baseline models. Unlike the other models, TACT successfully combines contrastive learning with medical hierarchies, allowing it to track precise disease trajectories and discover clinically actionable patient phenotypes. TACT establishes a comprehensive framework for characterizing patient heterogeneity as it enables the identification of subgroups with distinct disease progression profiles.