Skip to yearly menu bar Skip to main content


Explanation-based Data Augmentation for Image Classification

Sandareka Wickramanayake · Wynne Hsu · Mong Li Lee


Keywords: [ Deep Learning ] [ Vision ] [ Interpretability ] [ Machine Learning ]


Existing works have generated explanations for deep neural network decisions to provide insights into model behavior. We observe that these explanations can also be used to identify concepts that caused misclassifications. This allows us to understand the possible limitations of the dataset used to train the model, particularly the under-represented regions in the dataset. This work proposes a framework that utilizes concept-based explanations to automatically augment the dataset with new images that can cover these under-represented regions to improve the model performance. The framework is able to use the explanations generated by both interpretable classifiers and post-hoc explanations from black-box classifiers. Experiment results demonstrate that the proposed approach improves the accuracy of classifiers compared to state-of-the-art augmentation strategies.

Chat is not available.