Timezone: »
Neural networks embedded in safety-sensitive applications such as self-driving cars and wearable health monitors rely on two important techniques: input attribution for hindsight analysis and network compression to reduce its size for edge-computing. In this paper, we show that these seemingly unrelated techniques conflict with each other as network compression deforms the produced attributions, which could lead to dire consequences for mission-critical applications. This phenomenon arises due to the fact that conventional network compression methods only preserve the predictions of the network while ignoring the quality of the attributions. To combat the attribution inconsistency problem, we present a framework that can preserve the attributions while compressing a network. By employing the Weighted Collapsed Attribution Matching regularizer, we match the attribution maps of the network being compressed to its pre-compression former self. We demonstrate the effectiveness of our algorithm both quantitatively and qualitatively on diverse compression methods.
Author Information
Geondo Park (Korea Advanced Institute of Science and Technology)
June Yong Yang (Korea Advanced Institute of Science and Technology)
Sung Ju Hwang (KAIST, AITRICS)
Eunho Yang (Korea Advanced Institute of Science and Technology; AItrics)
More from the Same Authors
-
2022 : Distortion-Aware Network Pruning and Feature Reuse for Real-time Video Segmentation »
Hyunsu Rhee · Dongchan Min · Sunil Hwang · Bruno Andreis · Sung Ju Hwang -
2022 : Targeted Adversarial Self-Supervised Learning »
Minseon Kim · Hyeonjeong Ha · Sooel Son · Sung Ju Hwang -
2022 : Few-Shot Transferable Robust Representation Learning via Bilevel Attacks »
Minseon Kim · Hyeonjeong Ha · Sung Ju Hwang -
2023 Poster: GEX: A flexible method for approximating influence via Geometric Ensemble »
SungYub Kim · Kyungsu Kim · Eunho Yang -
2023 Poster: Riemannian SAM: Sharpness-Aware Minimization on Riemannian Manifolds »
Jihun Yun · Eunho Yang -
2021 Poster: Adaptive Proximal Gradient Methods for Structured Neural Networks »
Jihun Yun · Aurelie Lozano · Eunho Yang -
2021 Poster: Unbiased Classification through Bias-Contrastive and Bias-Balanced Learning »
Youngkyu Hong · Eunho Yang -
2020 Poster: Bootstrapping neural processes »
Juho Lee · Yoonho Lee · Jungtaek Kim · Eunho Yang · Sung Ju Hwang · Yee Whye Teh -
2020 Poster: Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learning »
Jaehyung Kim · Youngbum Hur · Sejun Park · Eunho Yang · Sung Ju Hwang · Jinwoo Shin -
2020 Poster: Learning to Extrapolate Knowledge: Transductive Few-shot Out-of-Graph Link Prediction »
Jinheon Baek · Dong Bok Lee · Sung Ju Hwang -
2020 Poster: Time-Reversal Symmetric ODE Network »
In Huh · Eunho Yang · Sung Ju Hwang · Jinwoo Shin -
2020 Poster: Neural Complexity Measures »
Yoonho Lee · Juho Lee · Sung Ju Hwang · Eunho Yang · Seungjin Choi -
2020 Poster: Adversarial Self-Supervised Contrastive Learning »
Minseon Kim · Jihoon Tack · Sung Ju Hwang -
2020 Poster: MetaPerturb: Transferable Regularizer for Heterogeneous Tasks and Architectures »
Jeong Un Ryu · JaeWoong Shin · Hae Beom Lee · Sung Ju Hwang -
2020 Spotlight: MetaPerturb: Transferable Regularizer for Heterogeneous Tasks and Architectures »
Jeong Un Ryu · JaeWoong Shin · Hae Beom Lee · Sung Ju Hwang -
2020 Poster: Few-shot Visual Reasoning with Meta-Analogical Contrastive Learning »
Youngsung Kim · Jinwoo Shin · Eunho Yang · Sung Ju Hwang -
2018 Poster: Uncertainty-Aware Attention for Reliable Interpretation and Prediction »
Jay Heo · Hae Beom Lee · Saehoon Kim · Juho Lee · Kwang Joon Kim · Eunho Yang · Sung Ju Hwang -
2018 Poster: Joint Active Feature Acquisition and Classification with Variable-Size Set Encoding »
Hajin Shim · Sung Ju Hwang · Eunho Yang -
2018 Poster: DropMax: Adaptive Variational Softmax »
Hae Beom Lee · Juho Lee · Saehoon Kim · Eunho Yang · Sung Ju Hwang -
2015 Poster: Closed-form Estimators for High-dimensional Generalized Linear Models »
Eunho Yang · Aurelie Lozano · Pradeep Ravikumar -
2015 Spotlight: Closed-form Estimators for High-dimensional Generalized Linear Models »
Eunho Yang · Aurelie Lozano · Pradeep Ravikumar -
2015 Poster: Robust Gaussian Graphical Modeling with the Trimmed Graphical Lasso »
Eunho Yang · Aurelie Lozano -
2014 Poster: Elementary Estimators for Graphical Models »
Eunho Yang · Aurelie Lozano · Pradeep Ravikumar -
2013 Poster: Conditional Random Fields via Univariate Exponential Families »
Eunho Yang · Pradeep Ravikumar · Genevera I Allen · Zhandong Liu -
2013 Poster: On Poisson Graphical Models »
Eunho Yang · Pradeep Ravikumar · Genevera I Allen · Zhandong Liu -
2013 Poster: Dirty Statistical Models »
Eunho Yang · Pradeep Ravikumar -
2012 Poster: Graphical Models via Generalized Linear Models »
Eunho Yang · Pradeep Ravikumar · Genevera I Allen · zhandong Liu -
2012 Oral: Graphical Models via Generalized Linear Models »
Eunho Yang · Pradeep Ravikumar · Genevera I Allen · zhandong Liu