Bayesian Neural Networks for Stable Feature Selection in High-Dimensional Learning

Feature selection in high-dimensional, low-sample-size settings is challenging when predictive relationships are nonlinear and uncertainty in selected features is important. We propose weight-sharing Bayesian Neural Networks (wsBNN), which combine Bayesian neural networks with structured feature-level sparsity through shared spike-and-slab priors. A shared inclusion variable controls all connections originating from each input feature, enabling direct and interpretable feature selection while retaining nonlinear modeling capacity. We use scalable variational inference to estimate the posterior and establish consistency conditions for the resulting variational posterior. Experiments on simulated and benchmark datasets demonstrate stable feature selection with competitive predictive performance, while application to TCGA-BRCA gene-expression data identifies biologically meaningful features and pathways.