Beyond Graph Serialization: Topology-Aware Low-Rank Operators for Small Language Models

Small language models (SLMs) offer an attractive basis for compact AI systems, but graph-structured inputs are commonly exposed to them only through text serialization, forcing the model to reconstruct topology indirectly from token order. We introduce TopoOp-SLM, a lightweight structural adaptation that preserves a standard serialized language-model interface while injecting explicit graph computation into selected decoder layers. Graph-sparse, low-rank residual operators pool node-span representations, propagate messages along directed edges, and scatter updates back to the corresponding token spans. The frozen 1.5B backbone and vocabulary prediction head remain unchanged; the topology pathway adds only 0.60M trainable parameters on top of a 2.18M-parameter QLoRA adaptation. Across five real-world labeled property graph tasks, TopoOp-SLM consistently improves Macro-F1, balanced accuracy, and MCC over an input-matched serialized SLM. Post-training interventions further show that the added pathway is active and sensitive to the supplied adjacency. The results suggest that compact language models can acquire useful graph-structural inductive bias through small in-place architectural additions rather than a separate graph encoder or a larger backbone.