FlexBench: In Silico and In Vitro Benchmarking of Structure-Based AI for Small Molecule Drug Design

Structure-based AI methods for small molecule drug discovery (SMDD) use AI to design molecules that bind to a protein given its 3D structure, with the goal of accelerating the discovery of new small molecule therapeutics. However, these methods have not been carefully benchmarked against each other or traditional methods for SMDD. To fill this gap, we present FlexBench, a benchmark of recent structure-based AI methods for SMDD, including our internal model SAGE-Flex and BoltzMol-1, compared to docking-based methods on four internal GPCR targets. Unlike prior benchmarks, FlexBench holds the compute budget and chemical space constant, making differences in performance attributable to the methods rather than their resources. Furthermore, we evaluated designs not just in silico but also in vitro, with 276 compounds synthesized and tested, which is one to two orders of magnitude more than typical prospective studies. While all methods had tradeoffs across the in silico metrics, the structure-based AI methods designed molecules in regions of chemical space that were unexplored by docking-based methods, thereby unlocking new molecular scaffolds. Of these methods, SAGE-Flex best combined that unique chemistry with a strong in silico profile, resulting in a hit rate of 12% in vitro versus 3% for BoltzMol-1. FlexBench thus effectively evaluates the strengths and weaknesses of structure-based AI methods for SMDD.