A General Filter-Enhanced Approach to Smartphone Hyperspectral Imaging

Hyperspectral reconstruction from RGB imagery offers an affordable alternative to costly hyperspectral acquisition devices. However, its accuracy is fundamentally constrained by the limited spectral resolution of standard RGB cameras. In this paper, we propose new approach that leverage the multi-camera systems integrated into most modern smartphones and enhance their capabilities by modulating their spectral response functions using carefully selected spectral filters, enabling a smartphone to capture richer spectral information. We propose a neural network able to exploit auxiliary RGB images with diverse spectral characteristics, enabling more accurate hyperspectral reconstruction compared to conventional single-image methods. Training is supported by a newly collected dataset, Doomer, which consists of multiple misaligned RGB images alongside corresponding hyperspectral data. Finally, our method generalizes across different smartphone devices through a easy to implement calibration procedure, ensuring practical applicability and scalability.