Timezone: »

 
Learning from uncertain concepts via test time interventions
Ivaxi Sheth · Aamer Abdul Rahman · Laya Rafiee Sevyeri · Mohammad Havaei · Samira Ebrahimi Kahou

@
in Trustworthy and Socially Responsible Machine Learning »
Event URL: https://openreview.net/forum?id=WVe3vok8Cc3 »

With neural networks applied to safety-critical applications, it has become increasingly important to understand the defining features of decision-making. Therefore, the need to uncover the black boxes to rational representational space of these neural networks is apparent. Concept bottleneck model (CBM) encourages interpretability by predicting human-understandable concepts. They predict concepts from input images and then labels from concepts. Test time intervention, a salient feature of CBM, allows for human-model interactions. However, these interactions are prone to information leakage and can often be ineffective inappropriate communication with humans. We propose a novel uncertainty based strategy, \emph{SIUL: Single Interventional Uncertainty Learning} to select the interventions. Additionally, we empirically test the robustness of CBM and the effect of SIUL interventions under adversarial attack and distributional shift. Using SIUL, we observe that the interventions suggested lead to meaningful corrections along with mitigation of concept leakage. Extensive experiments on three vision datasets along with a histopathology dataset validate the effectiveness of our interventional learning.

[ Poster]  [ OpenReview]  [ Topia] 

Author Information

Ivaxi Sheth (Imperial College London)
Aamer Abdul Rahman (Universite de Montreal)
Laya Rafiee Sevyeri (Concordia University)

I am second year PhD student working on Deep learning problems.

Mohammad Havaei (Imagia)
Samira Ebrahimi Kahou (McGill University)

More from the Same Authors