Institutional Confounding in Background-Based Acquisition Detection for Medical Imaging

Machine learning models deployed in clinical settings encounter images acquired under conditions absent from their training data, failing silently without a drop in reported confidence. For example, when we trained a lesion classifier on dermoscopic images and evaluated it on consumer smartphone photographs, the accuracy dropped from 81.8\% to 33.9\%, with melanoma sensitivity collapsing from 67.1\% to 1.9\%. In our experiments, standard out-of-distribution (OOD) detectors applied to the diagnostic model reach only 62–72\% accuracy due to high in-distribution baseline uncertainty. An alternative safeguard, classifying the acquisition modality directly from the background after masking the segmented lesion, achieves 96.0\% accuracy on an isolated dataset pair. However, deployment requires cross-institutional transfer. We assembled six sources spanning five institutions (8021 images): the dermoscopic HAM10000 (Vienna), ISIC~2019, and Derm7pt-dermoscopic (Vancouver), and the non-dermoscopic PAD-UFES-20 (Brazil), SCIN (crowdsourced), and Derm7pt-clinical (Vancouver). Derm7pt contributes to both classes by imaging every lesion, both dermoscopically and clinically, while keeping the institution, patient, and pathology constant. Lesions are segmented via Otsu thresholding and masked, and background features from a frozen encoder are classified using gradient-boosted decision trees. We benchmark hand-crafted acquisition-physics descriptors against supervised and self-supervised deep representations across leave-two-source-out and leave-one-institution-out protocols. The underlying acquisition signal is strong: across 1,010 matched image pairs, clinical photographs score higher on non-dermoscopic classification in 79.6\% of pairs (sign test $p = 5.8 \times 10^{-84}$), separating modalities at AUROC 0.984-0.989 while holding the institution constant. However, transfer fails: mean accuracy on unseen sources drops to 68.9\%. When one institution supplies both classes, performance degrades below chance (AUROC 0.218), yielding inverted predictions. Normalizing file encodings and applying oracle threshold selection confirm that this failure is not driven by provenance artifacts or threshold miscalibration. Rather, models exploit background institutional signatures (84.7-94.4\% six-way source classification versus 16.7\% chance) whenever source and modality correlate during training. Self-supervised features optimized via worst-group loss achieve 90.9\% mean and 82.4\% worst-case accuracy on held-out institutions, compared to 77.9\% and 54.3\% for supervised features under standard empirical risk minimization. Probing indicates that institutional information remains decodable, consistent with findings on residual information in adversarially scrubbed representations. While prior work treats site signatures as diagnostic confounds, here the acquisition characteristic is the explicit prediction target. Because equipment and institutions co-vary systematically in medical imaging, OOD safeguards require institution-level leave-out benchmarks reported under worst-case rather than mean metrics. By exposing a failure mode that dataset-level evaluation conceals, our benchmark and protocol give developers a concrete tool for detecting institutional shortcut learning before deployment.