Delta-SSM: Exact parallelization of spiking State Space Models for Efficient Sequence Modeling

State Space Models (SSMs) have emerged as an efficient alternative for sequence modeling. However, their deployment on edge hardware remains limited by dense matrix multiplications, which make it difficult to satisfy strict computational constraints. Spiking Neural Networks (SNNs) address this issue through event-driven computation, leveraging sparse spiking activity to reduce computational cost. However, existing spiking SSM variants inherit the sequential dynamics of SNNs, hindering parallelization and thereby limiting training efficiency and scalability. To address these limitations, we propose Delta-SSM, a spiking SSM variant based on the Mamba-2 architecture. By reducing redundant operations across time, Delta-SSM lowers computational cost and enables event-based computation. This is achieved through a custom integer spiking neuron whose sequential dynamics admit an exact time-parallel formulation that reduces to activation quantization. This enables fully parallel training without approximating the underlying spiking model. We evaluate Delta-SSM on a range of audio and language tasks, showing that it achieves approximately 45% of the original computational cost with limited performance degradation.