Performance Matching Does Not Imply Representational Equivalence in LoRA and Full Encoder Find-Tuning

Low-Rank Adaptation (LoRA) can match full encoder fine-tuning (Full FT) on downstream accuracy, but whether matched behavior implies matched internal representations is unclear. We test this directly in a controlled dSprites convolutional autoencoder. From shared checkpoints trained on scale and position, we adapt to held-out shape and symmetry-aware orientation with either LoRA or full encoder fine-tuning, keeping the decoder frozen. We compare representations only for LoRA checkpoints within 0.02 of their Full FT reference on both mean adaptation accuracy and each factor accuracy; this tolerance is fixed in advance, unmatched checkpoints are excluded, and representation metrics do not influence checkpoint selection. Among matched checkpoints, LoRA and Full FT diverge in activation-space geometry at bottleneck widths 16 and 32: LoRA has lower participation ratio by 0.491 ± 0.578 and 1.145 ± 0.358 and lower effective rank by 0.970 ± 0.490 and 2.671 ± 0.584, respectively, with the same direction across all three confirmatory seeds. At width 32, these dimensionality differences occur in all 9 matched rank-by-seed comparisons, which share three Full FT references. Factor interference is also higher for LoRA, by 0.0060 ± 0.0025 at width 16 and 0.0064 ± 0.0033 at width 32. This increased interference occurs in 8 of the 9 width-32 matched comparisons, although the absolute shifts remain small relative to the 1/d scale expected for independent random directions. Width 8 behaves differently. Participation ratio is slightly higher for LoRA, and effective-rank differences are near zero and inconsistent, showing that the geometric divergence depends on bottleneck capacity. Separately, rank-2 LoRA never reaches the matched-performance regime at any width. With three seeds, we report effect magnitudes and directional consistency rather than significance tests. Matched downstream performance therefore does not guarantee matched representation geometry.