A Representation Problem: Discretization Enables Band Gap Steering in Crystal-Generating LLMs

Language models have shown to be promising tools for crystal generation. However, while many of these models are trained on crystal files that contain properties, the objective of many models lies in the validity and novelty of the crystals they produce. Within this work, we focus on improving both understanding and steering ability of language models for crystal generation. We sweep band gap from 1 to 5 eV and measure the realized band gap of the generated structures. After demonstrating the weakness of current models, we then restructure the format of the property within training text. Encoding the band gap as a two-decimal number gives a steering slope of $0.009 \pm 0.015$ eV/eV: the output distribution does not move as the request changes. Rounding the band gap to integer bins and adding an auxiliary bin-classification loss gives $0.659 \pm 0.014$ eV/eV over five targets. Across three base models we find that scale increases the model's ability, CIF midtraining mainly buys structure-prediction accuracy, and the representation has the greatest impact on the ability to steer.