BitShift-RoPE: Zero-FLOP Relative Positional Encoding for Spiking Neural Network Transformers

Spiking Neural Networks (SNNs) provide energy-efficient computation by utilizing binary spike-driven additions, but applying transformer architectures remains challenging due to the lack of an SNN-compatible positional encoding (PE). Current PE methods either rely on floating-point arithmetic, which destroys the fundamental binary nature of SNNs, or fail to accurately capture relative spatial distances. To overcome this, we introduce Bit Shift Rotary Position Embedding (BitShift-RoPE), the first relative PE framework that fully preserves the binary integrity of SNN queries and keys. By replacing the floating-point trigonometric rotations in standard RoPE with discrete cyclic-shift operations via hardware memory-pointer routing, our method achieves zero-FLOP position encoding. We analytically map multi-scale exponential decay frequencies into integer shift steps and theoretically extend the formulation to 2D orthogonal spaces to capture complex spatial geometries. Crucially, because the binary integrity of the queries and keys is strictly maintained, the subsequent attention calculations rely solely on sparse bitwise AND operations. Our evaluations across time-series forecasting (avg $R^2$ 0.758), text classification (69.91\% accuracy), and image classification benchmarks (84.17\% accuracy) demonstrate that BitShift-RoPE achieves state-of-the-art performance while consuming the same energy as the vanilla SNN backbone (0.216 mJ/sample, identical to vanilla Spikformer). The code is available at \url{https://anonymous.4open.science/r/Bit\_Shift\_RoPE-16D2/}.