PMA-Diffusion: A Physics-guided Mask Aware Diffusion Framework for Traffic State Estimation from Sparse Observations

High-resolution Traffic State Estimation (TSE) is a foundational key research topic for building efficient, safe, reliable, and resilient transportation and mobility systems in smart cities. It can be used for various Intelligent Transportation Systems (ITS) applications such as Advanced Transportation Management Systems (ATMS) and Advanced Traveler Information Systems (ATIS). Yet, in practice, urban sensing infrastructures for the transportation and mobility systems, such as loop detectors and probe vehicles, provide data that is sparse, noisy, and unevenly distributed across city networks, limiting their utility for real-time decision-making and long-term planning. We present PMA-Diffusion, a Physics-guided Mask-Aware Diffusion framework designed to reconstruct high-resolution traffic state from incomplete, sparse, noisy observations. PMA-Diffusion learns a mask-aware diffusion prior directly from sparse urban sensing data and employs an iterative posterior sampling that alternates denoising, observation replacement, and physics-guided projection steps. On the I-24 MOTION dataset with only 5\% sensor coverage, PMA-Diffusion outperformed physics-agnostic baselines and achieved nearly the performance of fully supervised models. By enabling accurate, high-resolution traffic state estimation in data-sparse environments, this work demonstrates how state-of-the-art AI methodologies can be applied to enhance the scalability and robustness of urban transportation and mobility systems. More broadly, our approach highlights a path toward applying physics-guided generative AI to other smart-city applications, such as energy grids, water distribution, and environmental monitoring, where sparse data remains a critical challenge but high-resolution information is essential for reliable, resilient, and sustainable smart city applications.