Is Saliency Really Captured By Gradient?

Numerous feature attribution (or saliency) measures have been proposed that utilise the gradients of the output with respect to features. Gradients in this setting unequivocally tell us about feature sensitivity by definition of the gradient, but do they really tell us about feature importance? We challenge the idea that sensitivity and importance are the same, and empirically show that gradients do not necessarily find important features that should be attributed to a models' prediction.