Same Label, Different Feature: Explainer Dependence and the Causal Validity of SAE Explanations

Automatically generated explanations of sparse-autoencoder (SAE) features are typically evaluated against activations, then read to select steering, monitoring, or unlearning handles. That use rests on an untested causal inference: that identically labelled features have similar causal effects. We introduce a dyadic intervention audit that tests this assumption by intervening on feature pairs assigned the same exact explanation and comparing their downstream-effect directions with same pool random pairs on identical contexts. Across four public explainers applied to the same 16,384 Gemma-2-2B layer-20 features, exact-collision rates (one explanation assigned to multiple features) range from 3.1% to 72.2%, and only 0.05–1.05% of one substantive explainer’s colliding pairs also collide under another. A post-hoc audit finds that label form accounts for a measurable part of this explainer dependence without establishing its cause. At layer 12, the raw same-label association is +0.265 matched-random SD (95% CI [+0.129, +0.401]), but prespecified adjustment for decoder geometry and activation rates reduces it to +0.075 SD ([−0.063, +0.205]). A frozen layer-20 measurement is also weaker at +0.057 SD ([−0.061, +0.175]), under a corpus draw that was not held fixed. Within this evaluated setting, shared labels prioritize candidates. They do not establish causal interchangeability. More broadly, each explainer defines its own measurement instrument, and explanation-based selection needs direct intervention tests before model control. Code: https://anonymous.4open.science/r/sae-collision-7ECE