Out-of-the-Box Tabular Foundation Models for Bayesian Optimization in Materials Discovery

Bayesian optimization constitutes the backbone of active learning loops for materials science. Gaussian processes serve as the de-facto surrogate model family, but require task-dependent kernel choices and can lack the modeling capacity for complex relationships. Tabular foundation models (TFMs) have recently emerged as the state-of-the-art for predictive tasks on structured data, without requiring bespoke kernels or hyperparameter tuning. We evaluated two TFMS, TabPFN and TabICL, as out-of-the-box surrogate models on two standard materials science benchmarks, comparing against a large field of baselines encompassing Gaussian processes, random forests, and CatBoost. Our findings indicate that TFMs are highly competitive surrogate models across all considered tasks, without having been trained for active learning, and while doing away with hyperparameter choices inherent to competing methods. We consider our results evidence towards the promise of pursuing the investigation and development of TFMs as a new paradigm for active learning in materials discovery.