Probabilistic Digital Twin for Data-driven Smart Weaning of Mechanical Circulatory Devices

In this paper, we study the sequential decision-making for smart weaning of mechanical circulatory (MCS) devices. MCS devices are percutaneous micro-axial flow pumps for the treatment of cardiogenic shock patients, by providing left ventricular unloading and forward flow of blood into the aorta. While clinical recommendations for the weaning of MCS devices exist, the strategy varies by care team, and data-driven approaches are limited. Offline reinforcement learning (RL) has proven to be successful in sequential decision-making tasks, but the prohibition of interactions with the patient as the environment constrains evaluating RL policies. This motivates the development of probabilistic digital twin models to simulate the environment. We propose a formulation for offline RL training and a probabilistic Transformer-based digital twin to model the noisy circulatory dynamics and evaluate offline RL policies. We show that our Transformer-based digital twin (TDT) achieves 35% lower error compared to baseline models. We also present a comprehensive benchmark on offline RL methods using TDT with clinically relevant metrics.