Contrastive Regulatory Embedding Attention for Personalized Differential Expression: Where Architecture Helps, and Where It Fails

Sequence-to-function (S2F) deep learning models predict gene expression, chromatin state, and variant effects from DNA, yet they consistently fail to predict expression differences between individuals: two personal genomes are almost identical, differing only at sparse single-nucleotide variants (SNVs) that mostly lie outside strongly constrained sequence, so the genetic variation separating two people is tiny relative to non-genetic and environmental noise. We ask whether targeted architectural changes to an Enformer backbone can close this gap. We introduce CREAM, a Contrastive Regulatory Embedding Attention Model that (i) retrieves multi-scale features at polymorphic bins to counter resolution loss from convolutional downsampling, (ii) contrasts paired individual sequences and predicts differential expression, and (iii) contextualizes local variants within sequence background via contrastive cross-attention. On noise-free simulations and on GTEx data across three tissues, CREAM significantly outperforms baselines on training genes, recovers tissue-specific expression, and its attention weights prioritize fine-mapped causal eQTLs. Despite this, generalization to unseen genes collapses to near-zero correlation for every method we tried, and an $L_1$ inductive bias that regularizes attention towards known causal eQTLs does not translate into better expression prediction. Diagnosing the failure, we find that (1) models memorize gene-specific regulatory context rather than learning transferable cis-grammar, (2) identifying causal variants is necessary but far from sufficient for quantitative expression prediction, and (3) single-run uncertainty is a reliable error proxy on training genes but breaks on unseen genes, where the model defaults to a confident zero for many cases. We identify the limits of architecture-level fixes for personalized gene regulation and diagnose why they fail to generalize.