Prism: Harmonizing Missing Modalities via Implicit Structural Alignment on Lightweight Pulse RWKV for Multimodal Crack Segmentation
Abstract
In multimodal crack segmentation for industrial facilities, the key challenge is preventing missing modalities from degrading pixel-level performance while keeping computation low. Existing methods struggle to harmonize missing modality effects and to efficiently, adaptively perceive cross-modal topology. We propose Prism, resilient to any modality miss, leveraging implicit structural alignment to harmonize missing data for high-quality segmentation at low computational cost. Prism comprises Modal Reconstructor (MoRe), Prompt-as-Knowledge Distillation (PaKD), a Sparse Gated Pulse Propagation Mixer (PulseMixer), and an Uncertainty-guided Connectivity Propagation Fusion module (UCPF). MoRe utilizes the manifolds of available modalities to probabilistically harmonize missing data, empowered by PaKD to master where to look via prompt distillation. PulseMixer efficiently breaks the static parameterization bottleneck and perceives anisotropic textures via Pulse Propagation Insight (PPI) and sparse gating. UCPF fuses cross-modal cues using uncertainty estimation to produce clear segmentation maps while suppressing background noise. Experiments on three multimodal crack datasets demonstrate strong performance under diverse modality-miss scenarios. On the depth dataset with 90\% missing depth modal, Prism achieves 0.8217 in F1 and 0.8478 in mIoU with only 2.62M parameters.