Rejected When Asked, Followed When Assumed: False Presuppositions in Medical Vision-Language Models

Many clinical questions assume a finding is present, for example by asking where it is or how extensive it is. We test whether a model that judges a finding absent when asked directly will still describe it when a separate question assumes it is present. It usually does. When the finding is absent and the model says so, five open models still describe it in 82.9% to 100.0% of cases, and a frontier reasoning system in 72.2%. For the open models, this rate stays at 84.9% or higher even when the model also detects the finding in the matched image. When the finding is present and the model says so, the open models describe it as asked in at least 95.5% of cases. Describing absent findings remains frequent even when the model is highly confident that the finding is absent, under other question wordings, and on clinician-authored high-risk cases. We next test whether the behavior can be reduced. Asking the model to verify first reduces this only partly, while telling it whether the finding is present has a much larger effect. Hallucination-focused fine-tuning can also reduce it, but it shifts the model's answers to the direct question at the same time. A lower error rate on false presuppositions therefore does not, by itself, show that a model has become better at judging whether a finding is present.