Skip to yearly menu bar Skip to main content


Efficient High-Order Interaction-Aware Feature Selection Based on Conditional Mutual Information

Alexander Shishkin · Anastasia Bezzubtseva · Alexey Drutsa · Ilia Shishkov · Ekaterina Gladkikh · Gleb Gusev · Pavel Serdyukov

Area 5+6+7+8 #127

Keywords: [ Sparsity and Feature Selection ] [ (Other) Applications ]


This study introduces a novel feature selection approach CMICOT, which is a further evolution of filter methods with sequential forward selection (SFS) whose scoring functions are based on conditional mutual information (MI). We state and study a novel saddle point (max-min) optimization problem to build a scoring function that is able to identify joint interactions between several features. This method fills the gap of MI-based SFS techniques with high-order dependencies. In this high-dimensional case, the estimation of MI has prohibitively high sample complexity. We mitigate this cost using a greedy approximation and binary representatives what makes our technique able to be effectively used. The superiority of our approach is demonstrated by comparison with recently proposed interaction-aware filters and several interaction-agnostic state-of-the-art ones on ten publicly available benchmark datasets.

Live content is unavailable. Log in and register to view live content