Reacting to Text Is Not Following It: Verifying Language Grounding in Driving VLAs

Natural language is increasingly used as the instrument of a safety evaluation. An evaluator changes an instruction, observes the resulting action, and reads the difference as evidence about the model. That reading holds only if the language is grounded in the behavior being measured, which is rarely checked. We introduce an environment-grounded audit that asks first whether the outcome metric can register change at all and whether the prompt reaches the policy, and only then whether concrete instructions, abstract objectives, and deployment context steer the car. Applied to three released driving vision-language-action models, it finds that each reacts to text without following it, and that each fails for a different reason. On Alpamayo, a training-deployment contrast that looks decisive on one scene collapses once a factorial control shows the objective has the same effect under both framings, and a hazardous instruction moves the car no more than a meaningless cue of the same length. The instruction is also missing from the model's own reasoning text, which separates failed uptake from refusal and leaves the emitted rationale unfaithful to the input. On SimLingo, a learned control token raises compliance by $58.65$ percentage points where the same request in ordinary English raises it by $4.55$ on average across four phrasings, so an evaluation written in English would understate what the model can be made to do by roughly a factor of thirteen. On AutoVLA, no scene shows a semantic contrast larger than the spread produced by paraphrasing the same instruction. A language-based safety result should therefore travel with the positive control showing the metric could have moved, the threshold declared before the run, and the point at which interpretation stops. Without those it is non-diagnostic rather than evidence of safety.