Poster

Progressive Feature Interaction Search for Deep Sparse Network

Chen Gao · Yinfeng Li · Quanming Yao · Depeng Jin · Yong Li

Keywords: [ Deep Learning ] [ Machine Learning ]

[ Abstract ]
[ OpenReview
Fri 10 Dec 8:30 a.m. PST — 10 a.m. PST

Abstract:

Deep sparse networks (DSNs), of which the crux is exploring the high-order feature interactions, have become the state-of-the-art on the prediction task with high-sparsity features. However, these models suffer from low computation efficiency, including large model size and slow model inference, which largely limits these models' application value. In this work, we approach this problem with neural architecture search by automatically searching the critical component in DSNs, the feature-interaction layer. We propose a distilled search space to cover the desired architectures with fewer parameters. We then develop a progressive search algorithm for efficient search on the space and well capture the order-priority property in sparse prediction tasks. Experiments on three real-world benchmark datasets show promising results of PROFIT in both accuracy and efficiency. Further studies validate the feasibility of our designed search space and search algorithm.

Chat is not available.