AI4Mat Program Committee

Rama Vasudevan - Oak Ridge National Lab

Maria Chan - Argonne National Lab

Vijay Narasimhan - Merck KGaA, Damstadt, Germany

Q & A with Sim2Mat Lightning Talk Panelists

We propose MatSci ML, a novel benchmark for modeling Materials Science using Machine Learning methods focused on solid-state materials with periodic crystal structures. Applying machine learning methods to solid-state materials is a nascent field with substantial fragmentation largely driven by the great variety of datasets used to develop machine learning models. This fragmentation makes comparing the performance and generalizability of different methods difficult, thereby hindering overall research progress in the field. Building on top of open-source datasets, including large-scale datasets like the OpenCatalyst, OQMD, NOMAD, the Carolina Materials Database, and Materials Project, the MatSci ML benchmark provides a diverse set of materials systems and properties data for model training and evaluation, including simulated energies, atomic forces, material bandgaps, as well as classification data for crystal symmetries via space groups. The diversity of properties in MatSci ML makes the implementation and evaluation of multi-task learning algorithms for solid-state materials possible, while the diversity of datasets facilitates the development of new, more generalized algorithms and methods across multiple datasets. In the multi-dataset learning setting, MatSci ML enables researchers to combine observations from multiple datasets to perform joint prediction of common properties, such as energy and forces. Using MatSci ML, we evaluate the performance of different graph neural networks and equivariant point cloud networks on several benchmark tasks spanning single task, multitask, and multi-data learning scenarios.

Practical applications of machine learning for materials discovery aimed at concrete industrial applications remain severely limited by the quantity and quality of the available data. Furthermore, little is known about the ability of machine learning models to extrapolate outside the training distribution, which is essential for the discovery of compounds with extraordinary properties. To address these challenges, we develop a novel deep representation learning framework for chemical compositions.The proposed model, named COmpositional eMBedding NETwork (CombNet), combines recent developments in graph-based encoding of chemical compositions with a supervised contrastive learning approach.This is motivated by the observation that contrastive learning can produce a regularized representation space from raw data, offering empirical benefits for extrapolation and low-data scenarios. Moreover, our method harnesses exclusively the chemical composition of the underlying materials, as structure is generally unavailable before the material is discovered.We demonstrate the effectiveness of CombNet over state-of-the-art methods under a bespoke evaluation scheme that simulates a realistic materials discovery scenario with experimental data.

Chemists employ a number of levels of abstraction for describing objects and communicating ideas. Most of this knowledge is in the form of natural language, through books, articles and oral explanations, due to its flexibility and capacity to connect the different levels of abstraction. Despite of this, machine-learning chemical models are typically limited to low-level abstractions like graph representations or dynamic point clouds that, although powerful, ignore important aspects like procedural details. In this work, we propose methods for exploring the chemical space at the rich level of natural language. In this setting, synthetic procedure paragraphs are split into segments in four possible classes, and are subsequently mapped into a latent space where they can be conveniently studied. We explore the structure of this space, and find interesting connections with experimental realisation that are beyond the scope of commonly used reaction SMILES. This work aims to draw a path towards LLM-based data processing and chemical space exploration, by analyzing chemical data in previously inaccessible ways that will ultimately allow for better understanding of materials design.

Experimental design for optimizing black-box functions is a fundamental problem in many science and engineering fields. In this problem, sample efficiency is crucial due to the time, money, and safety costs of real-world design evaluations. Existing approaches either rely on active data collection or access to large, labeled datasets of past experiments, making them impractical in many real-world scenarios. In this work, we address the more challenging yet realistic setting of few-shot experimental design, where only a few labeled data points of input designs and their corresponding values are available. We introduce Experiment Pretrained Transformers (ExPT), a foundation model for few-shot experimental design that combines unsupervised learning and in-context pretraining. In ExPT, we only assume knowledge of a finite collection of unlabelled data points from the input domain and pretrain a transformer neural network to optimize diverse synthetic functions defined over this domain. Unsupervised pretraining allows ExPT to adapt to any design task at test time in an in-context fashion by conditioning on a few labeled data points from the target task and generating the candidate optima. We evaluate ExPT on few-shot experimental design in challenging domains and demonstrate its superior generality and performance compared to existing methods. The source code is available at https://github.com/tung-nd/ExPT.git.

In the field of computational physics and material science, the efficient sampling of rare events occurring at atomic scale is crucial. It aids in understanding mechanisms behind a wide range of important phenomena, including protein folding, conformal changes, chemical reactions and materials diffusion and deformation. Traditional simulation methods, such as Molecular Dynamics and Monte Carlo, often prove inefficient in capturing the timescale of these rare events by brute force. In this paper, we introduce a practical approach by combining the idea of importance sampling with deep neural networks (DNNs) that enhance the sampling of these rare events. In particular, we approximate the variance-free bias potential function with DNNs which is trained to maximize the probability of rare event transition under the importance potential function. This method is easily scalable to high-dimensional problems and provides robust statistical guarantees on the accuracy of the estimated probability of rare event transition. Furthermore, our algorithm can actively generate and learn from any successful samples, which is a novel improvement over existing methods. Using a 2D system as a test bed, we provide comparisons between results obtained from different training strategies, traditional Monte Carlo sampling and numerically solved optimal bias potential function under different temperatures. Our numerical results demonstrate the efficacy of the DNN-based importance sampling of rare events.

We introduce a machine learning approach to determine the transition rates of silicon atoms on a single layer of carbon atoms, when stimulated by the electron beam of a scanning transmission electron microscope (STEM). Our method is data-centric, leveraging data collected on a STEM. The data samples are processed and filtered to produce symbolic representations, which we use to train a neural network to predict transition rates. These rates are then applied to guide a single silicon atom throughout the lattice to pre-determined target destinations. We present empirical analyses that demonstrate the efficacy and generality of our approach.

Absorption near edge structure in the core electron excitation spectra reflect the anisotropy of orbital at the transition final state and can be used for analyzing local atomic environment including its orientation. So far, the analysis of fine structures is mainly based on a fingerprint-matching with high-cost experimental or simulated spectra. If core electron excitation spectra including its anisotropy can be predicted at low cost using machine learning, the application range of the core electron excitation spectra will be accelerated and extended for such as orientation and electronic structure analysis of liquid crystals and organic solar cells at high spatial resolution. Here, we propose a message-passing neural network for predicting core electron excitation spectra using a unit direction vector in addition to molecular graphs as input.Using a database of calculated C K-edge spectra, we have confirmed that the network can predict core electron excitation spectra reflecting the anisotropy of molecules. Our model is expected to be expanded to other physical quantities in general that depend not only on molecular graphs but also on anisotropic vectors.

Structure determination is necessary to identify unknown organic molecules, such as those in natural products, forensic samples, the interstellar medium, and laboratory syntheses. Rotational spectroscopy enables structure determination by providing accurate 3D information about small organic molecules via their moments of inertia. Kraitchman analysis uses these moments to determine isotopic substitution coordinates, which are the unsigned $|x|,|y|,|z|$ coordinates of all atoms with natural isotopic abundance, including carbon, nitrogen, and oxygen. While unsigned substitution coordinates can verify guesses of structures, the missing $+/-$ signs make it a hard computational problem to determine the actual structure from just the substitution coordinates. To tackle this inverse problem, we develop KREED (Kraitchman REflection-Equivariant Diffusion), a diffusion generative model which infers a molecule's all-atom 3D structure conditioned on the molecular formula, moments of inertia, and unsigned substitution coordinates of carbon and other heavy atoms. KREED's top-1 predictions identify the correct 3D structure with $>$98\% accuracy on the QM9 and GEOM datasets when provided with substitution coordinates of all heavy atoms with natural isotopic abundance. When substitution coordinates are restricted to only a subset of carbons, accuracy is retained at 91\% for QM9 and 32\% for GEOM. On a test set of experimentally measured substitution coordinates gathered from the literature, KREED can identify the correct all-atom 3D structure in 25 of 33 cases, demonstrating experimental applicability for context-free 3D structure determination with rotational spectroscopy.

Emerald Cloud Lab, a cutting-edge platform, holds immense potential in revolutionizing scientific research by seamlessly integrating artificial intelligence (AI) technologies into laboratory workflows. This talk discusses how Emerald Cloud Lab stands as an ideal platform to support AI-assisted scientific research. With its innovative features, including remote access to state-of-the-art laboratory equipment and robust data management capabilities, researchers can leverage AI algorithms to accelerate experimentation, automate data analysis, and enhance the overall research process. This transformative synergy between AI and Emerald Cloud Lab not only expedites scientific discoveries but also fosters collaboration across geographical boundaries, ultimately advancing the frontiers of knowledge in diverse fields of science.

AI4Mat Poster Session

Fireside Chat on LLMs for Materials Design in conversation with AI4Mat Program Committee: - Andrew White (University of Rochester) - Gabe Gomes (Carnegie Mellon University) - Gowoon Cheon (Google DeepMind)

Deep learning models have drastically accelerated materials discovery by accelerating predictive computational simulations like density functional theory (DFT). Large open computational materials databases such as the Materials Project or OQMD contain O($10^6$) known structures, and it is now straightforward to search those databases for materials with exciting properties. However, these databases are limited to experimentally known materials or candidates discovered in high-throughput computational campaigns. Many state-of-the-art engineering advances in solar photovaltaics, battery electrodes, and catalysts are made by discovering materials with outstanding properties that have not yet been discovered. Generative models are a natural solution to expand families of interest through sampling. While popular methods are typically constructed from variational autoencoders or diffusion models, we propose fine-tuning large language models for generation of stable materials. While unorthodox, fine-tuning large language models on text-encoded atomistic data is simple to implement yet reliable, with around 90\% of sampled structures obeying physical constraints on atom positions and charges. Using energy of hull calculations from both learned ML potentials and gold-standard DFT calculations, we show that our strongest model (fine-tuned LLaMA-2 70B) can generate materials predicted to be metastable at about twice the rate (49\% vs 28\%) of CDVAE, a competing diffusion model. Because of text prompting's inherent flexibility, our models can simultaneously be used for unconditional generation of stable material, infilling of partial structures and text-conditional generation. Finally, we show that language models' ability to capture key symmetries of crystal structures improves with model scale, suggesting that the biases of pretrained LLMs are surprisingly well-suited for atomistic data.

We propose an instruction-based process for trustworthy data curation in materials science (MatSci-Instruct), which we then apply to finetune a LLaMa-based language model targeted for materials science (HoneyBee). MatSci-Instruct helps alleviate the scarcity of relevant, high-quality materials science textual data available in the open literature, and HoneyBee is the first billion-parameter language model specialized to materials science. In MatSci-Instruct we improve the trustworthiness of generated data by prompting multiple commercially available large language models for generation with an Instructor module (e.g. Chat-GPT) and verification from an independent Verifier module (e.g. Claude). Using MatSci-Instruct, we construct a dataset of multiple tasks and measure the quality of our dataset along multiple dimensions, including accuracy against known facts, relevance to materials science, as well as completeness and reasonableness of the data. Moreover, we iteratively generate more targeted instructions in a finetuning-evaluation-feedback loop leading to progressively better performance for our finetuned HoneyBee models. Our evaluation on the MatSci-NLP benchmark shows HoneyBee's outperformance of existing language models on materials science tasks and iterative improvement in successive stages of instruction refinement. We study the quality of HoneyBee's language modeling through automatic evaluation and analyze case studies to further understand the model's capabilities and limitations.

Recent progress in large transformers-based foundation models have demonstrated impressive capabilities in mastering complex chemical language representations. These models show promise in learning task-agnostic chemical language representations through a two-step process: pre-training on extensive unlabeled corpora and fine-tuning on specific downstream tasks. By utilizing self-supervised learning capabilities, foundation models have significantly reduced the reliance on labeled data and task-specific features, streamlining data acquisition and pushing the boundaries of chemical language representation. However, their practical implementation in further downstream tasks is still in its early stages and largely limited to sequencing problems. The proposed multimodal approach using MoLFormer, a chemical large language model, aims to demonstrate the capabilities of transformer based models to non-sequencing applications such as capturing design space of liquid formulations. Multimodal MoLFormer utilizes the extensive chemical information learned in pre-training from unlabeled corpora for predicting performance of battery electrolytes and showcases superior performance compared to state-of-the-art algorithms. The potential of foundation models in designing mixed material systems such as liquid formulations presents a groundbreaking opportunity to accelerate the discovery and optimization of new materials and formulations across various industries.

Generative models trained on internet-scale data are capable of generating novel and realistic texts, images, and videos. A natural next question is whether these models can advance science, for example by generating novel stable materials. Traditionally, models with explicit structures (e.g., graphs) have been used in modeling structural relationships in scientific data (e.g., atoms and bonds in crystals), but generating structures can be difficult to scale to large and complex systems. Another challenge in generating materials is the mismatch between standard generative modeling metrics and downstream applications. For instance, common metrics such as the reconstruction error do not correlate well with the downstream goal of discovering novel stable materials. In this work, we tackle the scalability challenge by developing a unified crystal representation that can represent any crystal structure (UniMat), followed by training a diffusion probabilistic model on these UniMat representations. Our empirical results suggest that despite the lack of explicit structure modeling, UniMat can generate high fidelity crystal structures from larger and more complex chemical systems, outperforming previous graph-based approaches under various generative modeling metrics. To better connect the generation quality of materials to downstream applications, such as discovering novel stable materials, we propose additional metrics for evaluating generative models of materials, including per-composition formation energy and stability with respect to convex hulls through decomposition energy from Density Function Theory (DFT). Lastly, we show that conditional generation with UniMat can scale to previously established crystal datasets with up to millions of crystals structures, outperforming random structure search (the current leading method for structure discovery) in discovering new stable materials.

Artificial intelligence holds promise to improve materials discovery. GFlowNets are an emerging deep learning algorithm with many applications in AI-assisted discovery. By using GFlowNets, we generate porous reticular materials, such as metal organic frameworks and covalent organic frameworks, for applications in carbon dioxide capture. We introduce a new Python package (matgfn) to train and sample GFlowNets. We use matgfn to generate the matgfn-rm dataset of novel and diverse reticular materials with gravimetric surface area above 5000 $m^2 /g$. We calculate single- and two-component gas adsorption isotherms for the top-100 candidates in matgfn-rm. These candidates are novel compared to the state-of-art ARC-MOF dataset and rank in the 90th percentile in terms of working capacity compared to the CoRE2019 dataset. We discover 15 materials outperforming all materials in CoRE2019.

Generative molecular design is becoming an increasingly valuable approach to accelerate materials discovery. Besides comparably small amounts of polymer data, also the complex higher-order structure of synthetic polymers makes generative polymer design highly challenging. We build upon a recent polymer representation that includes stoichiometries and chain architectures of monomer ensembles and develop a novel variational autoencoder (VAE) architecture encoding a graph and decoding a string. Most notably, our model learns a latent space (LS) that enables de-novo generation of copolymer structures including different monomer stoichiometries and chain architectures.

Accelerating material discovery holds the potential to greatly help mitigate the climate crisis. Discovering new solid-state crystals such as electrocatalysts, ionic conductors or photovoltaics can have a crucial impact, for instance, in improving the efficiency of renewable energy production and storage. In this paper, we introduce Crystal-GFlowNet, a generative model of crystal structures that sequentially samples a crystal's composition, space group and lattice parameters. This domain-inspired approach enables the flexible incorporation of physical and geometrical constraints, as well as the use of any available predictive model of a desired property as an objective function. We evaluate the capabilities of Crystal-GFlowNet by using as objective the formation energy of a crystal structure, as predicted by a new proxy model trained on MatBench. The results demonstrate that Crystal-GFlowNet is able to sample diverse crystals with low formation energy.

Equivariant graph neural networks force fields (EGraFFs) have shown great promise in modelling complex interactions in atomic systems by exploiting the graphs’ inherent symmetries. Recent works have led to a surge in the development of novel architectures that incorporate equivariance-based inductive biases alongside architectural innovations like graph transformers and message passing to model atomic interactions. However, thorough evaluations of these deploying EGraFFs for the downstream task of real-world atomistic simulations, is lacking. To this end, here we perform a systematic benchmarking of 6 EGraFF algorithms (NequIP, Allegro, BOTNet, MACE, Equiformer, TorchMDNet), with the aim of understanding their capabilities and limitations for realistic atomistic simulations. In addition to our thorough evaluation and analysis on eight existing datasets based on the benchmarking literature, we release two new benchmark datasets, propose four new metrics, and three new challenging tasks. The new datasets and tasks evaluate the performance of EGraFF to out-of-distribution data, in terms of different crystal structures, temperatures, and new molecules. Interestingly, evaluation of the EGraFF models based on dynamic simulations reveals that having a lower error on energy or force does not guarantee stable or reliable simulation or faithful replication of the atomic structures. Moreover, we find that no model clearly outperforms other models on all datasets and tasks. Importantly, we show that the performance of all the models on out-of-distribution datasets is unreliable, pointing to the need for the development of a foundation model for force fields that can be used in real-world simulations. In summary, this work establishes a rigorous framework for evaluating machine learning force fields in the context of atomic simulations and points to open research challenges within this domain.

The Solid-Electrolyte Interphase (SEI) formed in lithium-ion batteries is a vital but poorly-understood class of materials, combining organic and inorganic components. An SEI allows a battery to function by protecting electrode materials from unwanted side reactions. We use a combination of classical sampling and a novel machine learning model to produce the first set of SEI candidate structures ranked by predicted energy, to be used in future machine learning applications and compared to experimental results. We hope that this work will be the start of a more quantitative understanding of lithium-ion battery interphases and an impetus to development of machine learning models for battery materials.

The need to use a short time step is a key limit on the speed of molecular dynamics (MD) simulations. Simulations governed by classical potentials are often accelerated by using a multiple-time-step (MTS) integrator that evaluates certain potential energy terms that vary more slowly than others less frequently. This approach is enabled by the simple but limiting analytic forms of classical potentials. Machine learning interatomic potentials (MLIPs), in particular recent equivariant neural networks, are much more broadly applicable than classical potentials and can faithfully reproduce the expensive but accurate reference electronic structure calculations used to train them. They still, however, require the use of a single short time step, as they lack the inherent term-by-term scale separation of classical potentials. This work introduces a method to learn a scale separation in complex interatomic interactions by co-training two MLIPs. Initially, a small and efficient model is trained to reproduce short-time-scale interactions. Subsequently, a large and expressive model is trained jointly to capture the remaining interactions not captured by the small model. When running MD, the MTS integrator then evaluates the smaller model for every time step and the larger model less frequently, accelerating simulation. Compared to a conventionally trained MLIP, our approach can achieve a significant speedup (~3x in our experiments) without a loss of accuracy on the potential energy or simulation-derived quantities.

AI4Mat Program Committee

Inverse problems are central to material design. While numerous studies have focused on designing microstructures by inverting structure-property linkages for various material systems, such efforts stop short of providing realizable paths to manufacture such structures. Accomplishing the dual task of designing a microstructure and a feasible manufacturing pathway to achieve a target property requires inverting the complete process-structure-property linkage. However, this inversion is complicated by a variety of challenges such as inherent microstructure stochasticity, high-dimensionality, and ill-conditioning of the inversion. In this work, we propose a Bayesian framework leveraging a lightweight flow-based generative approach for the stochastic inversion of the complete process-structure-property linkage. This inversion identifies a solution distribution in the processing parameter space; utilizing these processing conditions realizes materials with the target property sets. Our modular framework readily incorporates the output of stochastic forward models as conditioning variables for a flow-based generative model, thereby learning the complete joint distribution over processing parameters and properties. We demonstrate its application to the multi-objective task of designing processing routes of heterogeneous materials given target sets of bulk elastic moduli and thermal conductivities.

We present a novel multimodal language model approach for predicting molecular properties by combining chemical language representation with physicochemical features. Our approach, Multimodal-MoLFormer, utilizes a causal multi-stage feature selection method that identifies physicochemical features based on their direct causal effect on a specific target property. These causal features are then integrated with the vector space generated by molecular embeddings from MoLFormer. In particular, we employ Mordred descriptors as physicochemical features and identify the Markov blanket of the target property, which theoretically contains the most relevant features for accurate prediction. Our results demonstrate a superior performance of our proposed approach compared to existing state-of-the-art algorithms, including the chemical language-based MoLFormer and graph neural networks, in predicting complex tasks such as biodegradability and PFAS toxicity estimation. Moreover, we demonstrate the effectiveness of our feature selection method in reducing the dimensionality of the Mordred feature space while maintaining or improving the model’s performance. Our approach opens up promising avenues for future research in molecular property prediction by harnessing the synergistic potential of both chemical language and physicochemical features, leading to enhanced performance and advancements in the field.

We present an equivariant neural network for predicting the phonon modes of the periodic crystals and molecules by evaluating the second derivative Hessian matrices of the energy model, which are first trained with the energy and force data. Such efficient Hessian prediction enables us to predict the phonon dispersion and the density of states for inorganic crystal material and can be fine-tuned with additional dataset. For molecules, we also derive the symmetry constraints for infrared/Raman active modes by analyzing the phonon mode irreducible representations. Our training paradigm further shows using Hessian as a new type of higher-order training data to improve the energy models beyond the lower-order energy and force data.

Given the vast spectrum of material properties characterizing each compound,learning representations for inorganic materials is intricate. The prevailing trendwithin the materials informatics community leans towards designing specializedmodels that predict single properties. We introduce a multi-task learning frame-work, wherein a transformer-based encoder is co-trained across diverse materialsproperties and a denoising objective, resulting in robust and generalizable mate-rials representations. Our method not only amplifies the performance observedin single-dataset pretraining, but also showcases scalability and adaptability to-ward multi-dataset pretraining. Experiments demonstrate that the trained encoderMTENCODER captures chemically meaningful representations, surpassing theperformance of contemporary structure-agnostic materials encoders. This approachpaves the way to improvements in a multitude of deep materials informatics tasks,prominently including materials property prediction and generation of synthesisroutes for novel materials discovery

The field of machine learning for materials property prediction and characterization is seeing rapid developments in models, datasets, and frameworks. While datasets and models grow in size, frameworks must mature concurrently to match the data requirements and quick development cycles required to support these growing workloads. The efficient training of models is one area where machine learning frameworks may be improved. Utilizing active learning querying strategies to train models from scratch using fewer data can lead to faster development cycles, model evaluations, and reduced costs of training. Well-studied active learning querying strategies from computer vision and natural language processing are directly applied to train an E(n)-GNN model from scratch using a subset of the Materials Project Database and Novel Materials Discovery (NOMAD) Database, with the results compared to data subset selection techniques and the standard training pipeline. In general, the models trained with active learning querying strategies meet or exceed the performance standard trained models while using significantly less training data.

Discovery of new materials has a documented history of propelling human progress for centuries and more. The behaviour of a material is a function of its composition, structure, and properties, which further depend on its processing and testing conditions. Recent developments in deep learning and natural language processing have enabled information extraction at scale from published literature such as peer-reviewed publications, books, and patents. However, this information is spread in multiple formats, such as tables, text, and images, and with little or no uniformity in reporting style giving rise to several machine learning challenges. Here, we discuss, quantify, and document these outstanding challenges in automated information extraction (IE) from materials science literature towards the creation of a large materials science knowledge base. Specifically, we focus on IE from text and tables and outline several challenges with examples. We hope the present work inspires researchers to address the challenges in a coherent fashion, providing to fillip to IE for the materials knowledge base.

Abstract Neural operators extend data-driven models to map between infinite-dimensional functional spaces. These models have successfully solved continuous dynamical systems represented by differential equations, viz weather forecasting, fluid flow, or solid mechanics. However, the existing operators still rely on real space, thereby losing rich representations potentially captured in the complex space by functional transforms. In this paper, we introduce a Complex Neural Operator (CoNO), that parameterizes the integral kernel in the complex fractional Fourier domain. Additionally, the model employing a complex-valued neural network along with aliasing-free activation functions preserves the complex values and complex algebraic properties, thereby enabling improved representation, robustness to noise, and generalization. We show that the model effectively captures the underlying partial differential equation with a single complex fractional Fourier transform. We perform an extensive empirical evaluation of CoNO on several datasets and additional tasks such as zero-shot super-resolution, evaluation of out-of-distribution data, data efficiency, and robustness to noise. CoNO exhibits comparable or superior performance to all the state-of-the-art models in these tasks. Altogether, CoNO presents a robust and superior model for modeling continuous dynamical systems, providing a fillip to scientific machine learning. Our code implementation is available at https://anonymous.4open.science/r/anonymous-cono.

Exploring the predictive capabilities of natural language processing models in material science is a subject of ongoing interest. This study examines material property prediction, relying on models to extract latent knowledge from compound names and material properties. We assessed various methods for contextual embeddings and explored pre-trained models like BERT and GPT. Our findings indicate that using information-dense embeddings from the third layer of domain-specific BERT models, such as MatBERT, combined with the context-average method, is the optimal approach for utilizing unsupervised word embeddings from material science literature to identify material-property relationships. The stark contrast between the domain-specific MatBERT and the general BERT model emphasizes the value of domain-specific training and tokenization for material prediction. Our research identifies a "tokenizer effect," highlighting the importance of specialized tokenization techniques to capture material names effectively during the pretraining phase. We discovered that a tokenizer which preserves compound names entirely, while maintaining a consistent token count, enhances the efficacy of context-aware embeddings in functional material prediction.

The use of machine learning for material property prediction and discovery has traditionally centered on graph neural networks that incorporate the geometric configuration of all atoms. However, in practice not all this information may be readily available, e.g.~when evaluating the potentially unknown binding of adsorbates to catalyst. In this paper, we investigate whether it is possible to predict a system's relaxed energy in the OC20 dataset while ignoring the relative position of the adsorbate with respect to the electro-catalyst. We consider SchNet, DimeNet++ and FAENet as base architectures and measure the impact of four modifications on model performance: removing edges in the input graph, pooling independent representations, not sharing the backbone weights and using an attention mechanism to propagate non-geometric relative information. We find that while removing binding site information impairs accuracy as expected, modified models are able to predict relaxed energies with remarkably decent MAE. Our work suggests future research directions in accelerated materials discovery where information on reactant configurations can be reduced or altogether omitted.

Surrogate machine learning models for expensive material simulations can be an effective method to estimate relevant properties, which can help reduce the number of experiments needed. However, significant difficulties can occur when attempting to learn from small simulated datasets, particularly for samples out of the domain of the training data. This work provides an exploration on training deep learning models on a dataset of 36 synthetic 3D equiaxed polycrystalline microstructures with different cubic textures with a focus on out-of-domain accuracy, analyzing a number of transfer learning set ups, domain adaptation methods, model architectures, and featurizations across two formulations of the problem. We develop an evaluation set-up to validate our results, and report several methods that provide better results than our baseline of a simple U-Net architecture.

Machine learning interatomic potentials (MLIPs) enable molecular simulations at longer time scales without compromising accuracy and at lower computational costs compared to electronic structure methods such as density functional theory (DFT). Application of MLIPs to complex functional-material development can help to create new scientific insights, however, MLIPs need ad-hoc training for each new system. Reaching sufficient accuracy through large-scale training is data-intensive, and requires a high level of technical proficiency from the user. Reliable MLIP construction requires an appropriate selection of representative structures and calibrated model uncertainty, while avoiding undersampling of the state space. Currently, there is a lack end-to-end automated software to take this complexity away from the end user. In this tutorial, we show how to use CURATOR, an open-source software-based autonomous batch active learning workflow. CURATOR trains message-passing graph neural networks and enables management of model training, production testing, data selection based on uncertainty estimation, optimal batch choice, labeling via DFT-based simulations, and retraining in a user-friendly way.

Machine learning is transforming materials discovery by providing rapid predictions of material properties, which enables large-scale screening for target materials. However, such models require training data. While automated data extraction from scientific literature has potential, current auto-generated datasets often lack sufficient accuracy and critical structural and processing details that influence the properties. Using band gap as an example, we demonstrate LLM-prompt-based extraction yields an order of magnitude lower error rate. Combined with additional prompts to select a subset of experimentally measured properties from pure, single-crystalline bulk materials, this results in an automatically extracted dataset that's larger and more diverse than the largest existing human-curated database of experimental band gaps. Finally, compared to the existing human-curated database, we show the model trained on our extracted database achieves a 15\% reduction in the mean absolute error of predicted band gaps.

Following the current trends for minimizing human intervention in training intelligent architectures, this paper proposes a self-supervised method for quality control of Additive Manufacturing (AM) parts. An Inconel 939 sample is fabricated with the Laser Powder Bed Fusion (L-PBF) method and scanned using X-ray Computed Tomography (XCT) to reveal the internal cracks. A self-supervised approach was adopted by employing three modules that generate crack-like features for training a CycleGAN network. The proposed method generates random cracks based on a combination of uniform and normal random variables and outperforms the others in fine-grain crack detection and capturing narrow tips. A preliminary investigation of the training process shows that the algorithm has the capability of predicting the crack propagation direction as well.

The demand for precise, data-efficient, and cost-effective exploration of chemical space has ignited growing interest in machine learning (ML), which exhibits remarkable capabilities in accelerating atomistic simulations of large systems over long time scales. Active learning is a technique widely used to reduce the cost of acquiring relevant ML training data. Here we present a modular, transferrable, and broadly applicable, parallel active learning orchestrator. Our workflow enables data and task parallelism for data generation, model training, and ML-enhanced simulations. We demonstrate its use in efficiently exploring multiple excited state potential energy surfaces and possible degradation pathways of an organic semiconductor used in organic light-emitting diodes. With our modular and adaptable workflow architecture, we expect our parallel active learning approach to be readily extended to explore other materials using state-of-the-art ML models, opening ways to AI-guided design and a better understanding of molecules and materials relevant to various applications, such as organic semiconductors or photocatalysts.

Machine learning (ML) techniques and atomistic modeling have rapidly transformed materials design and discovery. Specifically, generative models can swiftly propose promising materials for targeted applications. However, the predicted properties of materials through the generative models often do not match with calculated properties through ab initio calculations. This discrepancy can arise because the generated coordinates are not fully relaxed, whereas the many properties are derived from relaxed structures. Neural network-based potentials (NNPs) can expedite the process by providing relaxed structures from the initially generated ones. Nevertheless, acquiring data to train NNPs for this purpose can be extremely challenging as it needs to encompass previously unknown structures. This study utilized extended ensemble molecular dynamics (MD) to secure a broad range of liquid- and solid-phase configurations in metallic systems. Then, we could significantly reduce them through active learning without losing much accuracy. We found that the NNP trained from the distilled data could predict different energy-minimized closed-pack crystal structures even though those structures were not explicitly part of the initial data. Furthermore, the data can be translated to another metallic system without repeating the sampling and distillation processes. Our approach to data acquisition and distillation has demonstrated the potential to expedite the NNP development and enhance materials design and discovery by integrating generative models.

Discovering new materials is essential to solve challenges in climate change, sustainability and healthcare. A typical task in materials discovery is to search for a material in a database which maximises the value of a function. That function is typically expensive to evaluate, and often relies upon a simulation or an experiment. Here, we introduce SyMDis, a sample efficient optimisation method based on symbolic learning, that discovers near-optimal materials in a large database. SyMDis performs comparably to a state-of-the-art optimiser, whilst learning interpretable rules to aid physical and chemical verification. Furthermore, the rules learned by SyMDis generalise to unseen datasets and return high performing candidates in a zero-shot evaluation, which is difficult to achieve with other approaches.

Graph neural networks (GNNs) have been applied to a large variety of applications in materials science and chemistry. Here, we systematically investigate the graph construction for crystalline (periodic) materials and investigate its impact on the GNNs model performance. We propose the asymmetric unit cell as a representation to reduce the number of nodes needed to represent periodic graphs by exploiting all symmetries of the system. Without any loss in accuracy, this substantially reduces the computational cost and thus time needed to train large graph neural networks. Furthermore, with a systematically built GNN architecture based on message passing and line graph templates, we introduce a general architecture (Nested Graph Network, NGN) that is applicable to a wide range of tasks. Using those models, we improve state-of-the-art results across all tasks within the MatBench benchmark. Further analysis shows that optimized connectivity and deeper message functions are responsible for the improvement. Asymmetric unit cells and connectivity optimization can be generally applied to (crystal) graph networks, while our suggested nested graph framework will open new ways of systematic comparison of GNN architectures.

The ultimate ideal in AI-driven drug discovery is the automatic design of specific drugs for individual diseases, yet this goal remains technically distant at present. However, recent advancements in large language models (LLMs) have significantly broadened the scope of applications with various tasks being explored in the chemistry domain. To probe the potential of utilizing LLMs in drug discovery, we organized a contest: the LLM Drug Discovery Challenge. Participants were tasked with proposing molecular structures of active compound candidates for a designated drug target using LLM-based workflows. The proposed chemical structures were evaluated comprehensively through scoring by a panel of five judges with deep expertise in medicinal chemistry, structural biology, and computational chemistry. Nine participants tackled the challenge with their unique methodologies, exploring the possibilities and current limitations of leveraging LLMs in drug discovery. In this rapidly advancing field, we aim to discuss the directions of future developments and what is expected moving forward.

Self-driving laboratories (SDLs) consist of multiple stations that perform materialsynthesis and characterisation tasks. To minimize station downtime and maxi-mize experimental throughput, it is practical to run experiments in asynchronousparallel, in which multiple experiments are being performed at once in differ-ent stages. Asynchronous parallelization of experiments, however, introducesdelayed feedback (i.e. “pending points”), which is known to reduce Bayesianoptimizer performance. Here, we build a simulator for a multi-stage SDL and com-pare optimization strategies for dealing with delayed feedback and asynchronousparallelized operation. Using data from [1], we build a ground truth Bayesianoptimization simulator from 177 previously run experiments for maximizing theconductivity of functional coatings. We then compare search strategies such asnaive expected improvement, 4-mode exploration as proposed by the originalauthors and asynchronous batching. We evaluate their performance in terms ofnumber of stages, and short, medium and long-term optimization performance.Our simulation results showcase the trade-off between the asynchronous paralleloperation and delayed feedback.

Traditional molecular string representations, such as SMILES, often pose challenges for AI-driven molecular design due to their non-sequential depiction of molecular substructures. To address this issue, we introduce Sequential Attachment-based Fragment Embedding (SAFE), a novel line notation for chemical structures. SAFE reimagines SMILES strings as an unordered sequence of interconnected fragment blocks while maintaining full compatibility with existing SMILES parsers. It streamlines complex generative tasks, including scaffold decoration, fragment linking, polymer generation, and scaffold hopping, while facilitating autoregressive generation for fragment-constrained design, thereby eliminating the need for intricate decoding or graph-based models. We demonstrate the effectiveness of SAFE by training an 87-million-parameter GPT2-like model on a dataset containing 1.1 billion SAFE representations. Through extensive experimentation, we show that our SAFE-GPT model exhibits versatile and robust optimization performance. SAFE opens up new avenues for the rapid exploration of chemical space under various constraints, promising breakthroughs in AI-driven molecular design.

Bridging the time and length scales and the use of large molecular dynamics (MD) simulations in material science is expected to surge in the next few years, partially due to the development of highly accurate machine learning inter-atomic potentials that enable the simulation of multi-million atomic systems. We also expect a high demand for material science simulations using multiple nodes within high-performance computing facilities (HPCs) due to their computational intensity. Through the analysis of catalysis simulation setups consisting of bulk metallic systems with adsorbed molecular species on the surface, we identified various factors that affect parallel computing efficiency. To foster sustainable and ethical computing practices, this study employs the Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS) to find the optimal allocation of computing resources based on the simulation input. We thus propose guidelines to promote responsible computing within HPC architecture: Eco-Comp is a user-friendly automated Python tool that allows material scientists to optimize the power consumption of their simulations using one command. This tutorial gives a broad overview of the Eco-Comp software and its potential use for the material science community through an interactive guide.

Discovering new solid-state materials necessitates the ability to rapidly explore the vast space of crystal structures and locate stable regions. Generating stable materials with desired properties and composition is a challenging task because of (a) the exponentially large number of possibilities when the elements from the periodic table are considered along with vast variations in their 3D arrangement and corresponding lattice parameters and (b) the rarity of the stable structures. Furthermore, materials discovery requires not only optimized solution structures but also diversity in the configuration of generated material structures. Existing methods have difficulty when exploring large material spaces and generating significantly diverse samples with desired properties and requirements. We propose Crystal Hierarchical Generative Flow Network (CHGlownet), a new generative model that employs a hierarchical exploration strategy with Generative Flow Network to efficiently explore the material space while generating the crystal structure with desired properties. Our model decomposes the large material space into a hierarchy of subspaces of space groups, lattice parameters, and atoms. We significantly outperform the iterative generative methods such as Generative Flow Network (GFlowNet) and Physics Guided Crystal Generative Model (PGCGM) in crystal structure generative tasks in validity, diversity, and generating stable structures with optimized properties and requirements.

This tutorial describes a simulated laboratory for making use of reinforcement learning (RL) for chemical discovery. A key advantage of the simulated environment is that it enables RL agents to be trained safely and efficiently. In addition, it offer an excellent testbed for RL in general, with challenges which are uncommon in existing RL benchmarks. The simulated laboratory, denoted ChemGymRL, is open-soure, implemented according to the standard Gymnasium API, and is highly customizable. It supports a series of interconnected virtual chemical \emph{benches} where RL agents can operate and train. Within this tutorial introduce the environment, demonstrate how to train off-the-shelf RL algorithms on the benches, and how to modify the benches by adding additional reactions and other capabilities. In addition, we discuss future directions for ChemGymRL benches and RL for laboratory automation and the discovery of novel synthesis pathways. The software, documentation and tutorials are available here: \url{ur_ suppressed}.

This paper introduces MatKG-2, a Material Science knowledge graph autonomously generated through a Large Language Model (LLM) driven pipeline. Building on the groundwork of MatKG, MatKG-2 employs a novel 'committee of large language models' approach to extract and classify knowledge triples with an established ontology. Unlike the previous version, which relied on statistical co-occurrence, MatKG-2 offers more nuanced, ontology-based relationships. Using open LLMs such as Llama2 7b and Bloom 1b/7b, the study offers reproducibility and broad community engagement. By using 4-bit and 8-bit quantized versions for fine-tuning and inference, MatKG-2 is also more computationally tractable and therefore compatible with most commercially available GPUs. Our work highlights the potential of MatKG-2 in supporting Material Science data infrastructure and in contributing to the semantic web.

Metal-organic frameworks (MOFs) are of immense interest in applications such as gas storage and carbon capture due to their exceptional porosity and tunable chemistry. Their modular nature has enabled the use of template-based methods to generate hypothetical MOFs by combining molecular building blocks in accordance with known network topologies. However, the ability of these methods to identify top-performing MOFs is often hindered by the limited diversity of the resulting chemical space. In this work, we propose MOFDiff: a coarse-grained (CG) diffusion model that generates CG MOF structures through a denoising diffusion process over the coordinates and identities of the building blocks. The all-atom MOF structure is then determined through a novel assembly algorithm. As the diffusion model generates 3D MOF structures by predicting scores in E(3), we employ equivariant graph neural networks that respect the permutational and roto-translational symmetries. We comprehensively evaluate our model's capability to generate valid and novel MOF structures and its effectiveness in designing outstanding MOF materials for carbon capture applications with molecular simulations.

The application of machine learning models in chemistry has made remarkable strides in recent years. Even though there is considerable interest in automating common procedure in analytical chemistry using machine learning, very few models have been adopted into everyday use. Among the analytical instruments available to chemists, Nuclear Magnetic Resonance (NMR) spectroscopy is one of the most important, offering insights into molecular structure unobtainable with other methods. However, most processing and analysis of NMR spectra is still performed manually, making the task tedious and time consuming especially for large quantities of spectra. We present a transformer-based machine learning model capable of predicting the molecular structure directly from the NMR spectrum. Our model is pretrained on synthetic NMR spectra, achieving a top-1 accuracy of 67.0% when predicting the structure from both the $^1$H and $^{13}$C spectrum. Additionally, we train a model which, given a spectrum and a set of likely compounds, selects the structure corresponding to the spectrum. This model achieves a top-1 accuracy of 98.28% when trained on both $^1$H and $^{13}$C spectra in selecting the correct structure.

The available labeled data to support molecular property prediction are limited in size due to experimental time and cost requirements. However, unsupervised learning techniques can leverage vast databases of molecular structures, thus significantly expanding the scope of training data. We compare the effectiveness of pre-training data and modeling choices to support the downstream task of molecular aqueous solubility prediction. We also compare the global and local structure of the learned latent spaces to probe the properties of effective pre-training approaches. We find that the pre-training modeling choices affect predictive performance and the latent space structure much more than the data choices.

Understanding organic reaction mechanisms is crucial for interpreting the formation of products at the atomic and electronic level, but still remains as a domain of knowledgeable experts. The lack of a large-scale dataset with chemically reasonable mechanistic sequences also hinders the development of reliable machine learning models to predict organic reactions based on mechanisms as human chemists do. Here, we propose a method that automatically generates reaction mechanisms of a large dataset of organic reactions using autonomously extracted reaction templates and expert-coded mechanistic templates. By applying this method, we labeled 94.8\% of 33k USPTO reactions into chemically reasonable arrow-pushing diagrams, validated by expert chemists. Our method is simple, flexible, and can be expanded to cover a wider range of reactions, regardless of type or complexity. We envision it becoming an invaluable tool to propose reaction mechanisms, and to develop future reaction outcome prediction models and discover new reactions.

Navigating through the exponentially large chemical space to search for desirable materials is an extremely challenging task in material discovery. Recent developments in generative and geometric deep learning have shown promising results in molecule and material discovery but often lack evaluation with high-accuracy computational methods. This work aims to design novel and stable crystalline materials conditioned on a desired band gap. To achieve conditional generation, we: 1. Formulate crystal design as a sequential decision-making problem, create relevant trajectories based on high-quality materials data, and use conservative Q-learning to learn a conditional policy from these trajectories. To do so, we formulate a reward function that incorporates constraints for energetic and electronic properties obtained directly from density functional theory (DFT) calculations; 2. Evaluate the generated materials from the policy using DFT calculations for both energy and band gap; 3. Compare our results to relevant baselines, including a random policy, behavioral cloning, and unconditioned policy learning. Our experiments show that our conditioned policies achieve more targeted crystal structure designs and demonstrate the capability to perform crystal structure design evaluated with accurate and computationally expensive DFT calculations.

Utilizing computational methods to extract actional information from scientific data is essential due to the time-consuming and inaccurate nature of the manual processes of humans. To better serve the purpose, equipping computational methods with physical rules is necessary. Integrating deep learning models with symmetry awareness has emerged as a promising approach to significantly improve symmetry detection in experimental data, with techniques such as parameter sharing and novel convolutional layers enhancing symmetry recognition.[1,2,3,4,5,6] However, the challenge of integrating physical principles, such as symmetry, into these models persists. To address this, we have developed benchmarking datasets and training frameworks, exploring three perspectives to classify wallpaper group symmetries effectively. This endeavor aims to push the boundaries of deep learning models in comprehending symmetry and embed physical rules within them, ultimately unlocking new possibilities at the intersection of machine learning and physical symmetry, with valuable applications in materials science and beyond.

Machine learning has great potential to revolutionize experimental materials research; however, the degree to which these approaches accelerate novel discovery is rarely quantified. To this end, we propose a framework for characterizing the rate of “first discovery” of scientific hypotheses in the form of materials families. We use a combination of the SuperCon and Materials Project databases to simulate a scientific needle-in-a-haystack discovery problem as a motivating example. We use this approach to compare the ability of different adaptive sampling strategies to rediscover promising superconductor families, such as the Cuprates and iron-based superconductors. This methodology can be applied using various notions of novelty, making it applicable to discovery problems more broadly.

Efficient design of microstructures with targeted properties has always been a challenging task owing to the expensive and time-consuming nature of the problem. In recent years, generative models have been used to accelerate this process. However, most of these methods are hindered by the choice of their generative model - either due to stability and usability, like with GANs, or flexibility of the model itself, like the availability of a semantically meaningful latent space. We propose a diffusion autoencoder based generative design framework that not only provides the fidelity and stability benefits of diffusion models but also has a desirable latent space that can be exploited by evolutionary algorithms. We employ this framework to solve multiple simultaneous objectives to find a Pareto frontier of candidate microstructures. We also show that the search space of optimization can be drastically reduced by conditioning the model with target objective values. We demonstrate the efficacy of the proposed framework on a number of optimization and generative tasks based on two-phase morphology dataset derived from Cahn-Hilliard equations.

In order to fully utilise the potential throughput of automated synthesis and characterisation data collection, data analysis capabilities must have matching throughput, which consumes excessive (human) expert time even for small datasets.One such analysis task is unmixing; being able to generally separate, from a sample consisting of multiple components, the individual patterns characteristic of the constituent parts.Being able to do so quickly and reliably is important both for identifying samples containing unknown materials in large parallel batches (e.g. spray deposition) and for autonomous/closed loop refinement (e.g. flow synthesis).The problem can be akin to finding a needle in a haystack, where only a minuscule proportion of the many samples accessed by the automated synthesis contain some of the unknown in a small amount by mass, which may not even be proportionally reflected in the spectra.Even if patterns corresponding to each chemical component are known ahead of time, they are not trivial to separate, as they in fact change from sample to sample (e.g. peak shifting) due to small modifications in the component produced or processing conditions.Conventional approaches can be narrowly applicable (without severe modification or retraining) and suffer from excessive local minima.We propose instead a non-parametric approach based on exact optimal transport (OT) which allows for arbitrary variation through flexible patterns and better defined local minima.

Reinforcement learning (RL) over text representations can be effective for finding high-value policies that can search over graphs. However, RL requires careful structuring of the search space and algorithm design to be effective in this challenge. Through extensive experiments, we explore how different design choices for text grammar and algorithmic choices for training can affect an RL policy's ability to generate molecules with desired properties. We arrive at a new RL-based molecular design algorithm (ChemRLformer) and perform a thorough analysis using 25 molecule design tasks, including computationally complex protein docking simulations. From this analysis, we discover unique insights in this problem space and show that ChemRLformer achieves state-of-the-art performance while being more straightforward than prior work by demystifying which design choices are actually helpful for text-based molecule design.

Recent advances in Machine Learning-based DFT approximation have drastically accelerated computational adsorption energy estimation. New solutions to the climate crisis, however, hinges on the inverse material design problem: identifying new catalysts with precise desired adsorption energies for each target reaction. Here we introduce AdsorbRL, a Deep Reinforcement Learning (DRL) agent aiming to identify catalysts that best fit a target adsorption energy profile, trained using offline learning on the Materials Project and Open Catalyst 2020 data sets. While initial experiments with a DQN agent failed in a complex ternary compounds space, a simple Q-learning agent reaches near-optimal adsorption energy in element-wise traversal of the periodic table. Building on this insight, we introduce Random Edge Traversal to simplify the action space, and successfully train a single-objective DQN agent which improves target adsorption energy from random initial states by an average of 4.1 eV. We extend this approach to multi-objective, goal-conditioned learning, and train a DQN agent to identify materials with the highest (respectively lowest) adsorption energies possible for multiple simultaneous target adsorbates. We introduce a novel training scheme for this agent using objectives sub-sampling, and report experimental results which suggest improved performance in the multi-objective, goal-conditioned RL setup. Overall, our results demonstrate the strong potential of DRL agents to tackle the inverse catalyst design across complex chemical spaces.

In this paper, we present graph neural networks (GNNs) capturing anisotropic properties of polycrystals. Our submission fits the workshop topic of Machine learning algorithms for materials simulation. Our contributions include: (i) GNNs that feature a physics-inspired combination of the aggregation function and node attributes; (ii) case studies demonstrating excellent generalization of our GNNs to predicting anisotropic properties without the need in extensive training datasets.

A bottleneck in high-throughput nanomaterials discovery is the pace at which new materials can be structurally characterized. Although current machine learning (ML) methods show promise for the automated processing of electron diffraction patterns (DPs), they fail in high-throughput experiments where DPs are collected from crystals with random orientations. Inspired by the human decision-making process, a framework for automated crystal system classification from DPs with arbitrary orientations was developed. A convolutional neural network was trained using evidential deep learning, and the predictive uncertainties were quantified and leveraged to fuse multiview predictions. Using vector map representations of DPs, the framework achieves an unprecedented testing accuracy of 0.94 in the examples considered, is robust to noise, and retains remarkable accuracy using experimental data. This work highlights the ability of ML be used to accelerate experimental high-throughput materials data analytics.

Characteristic classes, which are abstract topological invariants associated with vector bundles, have become an important notion in modern physics with surprising real-world consequences. As a representative example, the incredible properties of topological insulators, which are insulators in their bulk but conductors on their surface, can be completely characterized by a specific characteristic class associated with their electronic band structure, the first Chern class. Given their importance to next generation computing and the computational challenge of calculating them using first-principles approaches, there is a need to develop machine learning approaches to predict the characteristic classes associated with a material system. To aid in this program we introduce the {\emph{Haldane bundle dataset}}, which consists of synthetically generated complex line bundles on the $2$-torus. We envision this dataset, which is not as challenging as noisy and sparsely measured real-world datasets but (as we show) still difficult for off-the-shelf architectures, to be a testing ground for architectures that incorporate the rich topological and geometric priors underlying characteristic classes.

Scanning probe spectroscopy generates high-dimensional data that is difficult to analyze in real time, hindering researcher creativity. Machine learning techniques like PCA accelerate analysis but are inefficient, sensitive to noise, and lack interpretability. We developed an unsupervised deep neural network constrained by a known empirical equation to enable real-time, robust fitting. Demonstrated on band-excitation piezoresponse force microscopy, our model fits cantilever response to a simple harmonic oscillators more than 4 orders of magnitude faster than least squares while enhancing robustness. It performs well on noisy data where conventional methods fail. Quantization-aware training enables sub-millisecond streaming inference on an FPGA, orders of magnitude faster than data acquisition. This methodology broadly applies to spectroscopic fitting and provides a pathway for real-time control and interpretation.

Property prediction plays an important role in material discovery. As an initial step to eventually develop a foundation model for material science, we introduce a new autoencoder called the MHG-GNN, which combines graph neural network (GNN) with Molecular Hypergraph Grammar (MHG). Results on a variety of property prediction tasks with diverse materials show that MHG-GNN is promising.

Metal-organic frameworks (MOFs), formed through coordination bonds between metal ions and organic ligands, are promising materials for efficient gas adsorption, due to their ultrahigh porosity, chemical tunability and large surface area. Because over a hundred thousand hypothetical MOFs have been reported to date, brute force discovery of the best performer MOF for a specific application is not feasible. Recently, predicting material properties using machine learning algorithms has played a crucial role in scanning large databases, but this often requires large labeled training sets, which is not always available. To address this, active learning, where the training set is constructed iteratively by querying only informative labels, is necessary. Moreover, in most cases, a very specific range of the property of interest is desirable. We employ a novel regression tree-based quantile active learning algorithm that uses partitions of a regression tree to select new samples to be added to the training set. It thereby limits the sample size while maximizing the prediction quality over a quantile of interest. Tests on benchmark MOF data sets demonstrate that focusing on a specific quantile is effective in learning regression models to predict electronic band gaps and CO$_2$ adsorption in the regions of interest, from a very limited labeled data set.

Prediction of average voltage of a cathode material, which is related to energy density, is an important task in a battery. However, it is difficult to develop a practical prediction model because relevant data is small, and important information including structure, regarded as a good modality for predicting properties of materials, is barely known except compositions. Inspired by these points, we propose a pretraining method utilizing a contrastive learning between compositions and structures(CLCS), which can improve the performance of voltage prediction task using only compositions of materials. First, we pretrained an composition encoder through contrastive learning between composition and structure representations, extracted by a transformer encoder and a graph neural network respectively, enabling the composition encoder to learn information associated with structures. Then, we transferred the composition encoder to a downstream task of predicting the average voltage with compositions. The performance of transferred model exceeds one of a model without pretraining by 9.7%. Also, with attention score analysis, we discovered that the transferred composition encoder focuses on lithium more than other elements in lithium-transition metal-oxygen systems compared to the composition encoder without pretraining.

We propose to utilize a neural network to build transition proposal distributions in simulated annealing (SA), which we use for combinatorial optimization on 2D-binary grids and thereby direct convergence towards states of structurally clustered patterns.To accomplish this we introduce a novel class of network architectures called template networks.A template network learns a template to construct a proposal distribution for state transitions of the stochastic process of the Metropolis algorithm, which forms the basis of SA.Each network represents a single constant pattern and is trained on the evaluation results of intermediate states of a single optimization run, resulting in an architecture not requiring an input layer.Using this learning scheme we equip the Metropolis algorithm with the ability to utilize information about past states, intentionally violating the Markov property of memorylessness, and therefore call our method Memory Metropolis (MeMe).Moreover, the emergence of structural clusters is encouraged by incorporating layers with limited local connectivity in the template network, while the network depth controls the learnable cluster sizes.By violating the Markov property and further dropping the consideration of transition properties when evaluating the Metropolis criterion, we deliberately bias the target distribution towards cluster formation.\Viewing the optimization objective of the Metropolis algorithm as a reward maximization links MeMe to deep reinforcement learning, where the policy is constructed from the discrepancy between the template and the current state.This allows to train the template network to find high-reward template-patterns.Detrimental actions (negative rewards) can be directly reverted by evaluating the Metropolis criterion which saves on computationally costly state evaluations.\We apply our algorithm to combinatorial optimization in nanophotonic inverse design and demonstrate that MeMe results in clustered design patterns suitable for direct optical chip fabrication which can not be found by plain SA or regularized SA. Code is available at https://XXXXXXXX.

We propose a multi-modal foundation model for small molecules, a shift from traditional AI models that are tailored for individual tasks and modalities. This model uses a late fusion strategy to align and fuse three distinct modalities: SELFIES, DFT properties, and optical spectrum. The model is pre-trained with over 6 billion samples to provide two primary functions, generating fused feature representations across the three modalities, and cross-modal predictions and genrations. As preliminary experiments, we demonstrate that the fused representation successfully improves the performance of property predictions for chromophore molecules, and showcase 6 distinct cross-modal inferences.

Organometallic complexes are ubiquitous in homogenous catalysis, and their optimization is of particular interest for many technologically relevant reactions. However, due to the large variety of possible metal-ligand and ligand-ligand interactions, finding the best combination of metal and ligands is an immensely challenging task. Here we present an inverse design framework based on a generative model for \textit{in-silico} design of such complexes. Given the importance of the spatial structure of a catalyst, the model directly operates on all-atom (including \ch{H}) representations in $3$D space. To handle the symmetries inherent to that data representation, it combines an equivariant diffusion model and an equivariant property predictor to drive sampling at inference time. We demonstrate the effectiveness of the proposed approach by designing catalysts for the Suzuki cross-coupling reaction, and validating a selection of novel proposed compounds with \textsc{DFT}.

In computational chemistry, crystal structure prediction (CSP) is an optimization problem that involves discovering the lowest energy stable crystal structure for a given chemical formula. This problem is challenging as it requires discovering globally optimal designs with the lowest energies on complex manifolds. One approach to tackle this problem involves building simulators based on density functional theory (DFT) followed by running search in simulation, but these simulators are painfully slow. In this paper, we study present and study an alternate, data-driven approach to crystal structure prediction: instead of directly searching for the most stable structures in simulation, we train a surrogate model of the crystal formation energy from a database of existing crystal structures, and then optimize this model with respect to the parameters of the crystal structure. This surrogate model is trained to be conservative so as to prevent exploitation of its errors by the optimizer. To handle optimization in the non-Euclidean space of crystal structures, we first utilize a state-of-the-art graph diffusion auto-encoder (CD-VAE) to convert a crystal structure into a vector-based search space and then optimize a conservative surrogate model of the crystal energy, trained on top of this vector representation. We show that our approach, dubbed LCOMs (latent conservative objective models), performs comparably to the best current approaches in terms of success rate of structure prediction, while also drastically reducing computational cost.

Sampling diverse, thermodynamically feasible molecular conformations plays a crucial role in predicting properties of a molecule. In this paper we propose to use GFlowNet for sampling conformations of small molecules from the Boltzmann distribution, as determined by the molecule's energy. The proposed approach can be used in combination with energy estimation methods of different fidelity and discovers a diverse set of low-energy conformations for highly flexible drug-like molecules. We demonstrate that GFlowNet can reproduce molecular potential energy surfaces by sampling proportionally to the Boltzmann distribution.

Atomic-scale imaging of 2D and quantum materials benefits from precisely extracting crystallographic strain, shear, and rotation to understand their mechanical, optical and electronic properties. One powerful technique is 4D-TEM (4-dimensional transmission electron microscopy), where a convergent electron beam is scanned across a sample while measuring the resulting diffraction pattern with a direct electron detector. Extracting the crystallographic strain, shear, and rotation from this data relies either on cross-correlation of probe templates (e.g., implemented in py4DSTEM) or determining the center of mass (CoM) of the diffraction peaks. These algorithms have limitations. They require manual preprocessing and hyperparameter tuning, are sensitive to signal-to-noise, and generally are difficult to automate. There is no one-size-fits-all algorithm.Recently, machine learning techniques have been used to assist in analyzing 4D-TEM data, however, these models do not possess the capacity to learn the strain, rotation, or translation instead they just learn an approximation that almost aways tends to be correct as long as the test examples are within the training dataset distribution.We developed a novel neural network structure – Cycle Consistent Spatial Transforming Autoencoder (CC-ST-AE). This model takes a set of diffraction images and trains a sparse autoencoder to classify an observed diffraction pattern to a dictionary of learned “averaged” diffraction patterns. Secondly, it learns the affine transformation matrix parameters that minimizes the reconstruction error between the dictionary and the input diffraction pattern. Since the affine transformation includes translation, strain, shear, and rotation, we can parsimoniously learn the strain tensor. To ensure the model is physics conforming, we train the model cycle consistently, by ensuring the inverse affine transformation from the dictionary results in the original diffraction pattern.We validated this model on a number of benchmark tasks including: A Simulated 4D TEM data of $WS_2$ and $WSe_2$ lateral heterostructures (noise free) with a ground truth of the strain, rotation and shear parameters. Secondly, we test this model on experimental 4D TEM on 2D heterostructures of tungsten disulfide ($WS_2$) and tungsten diselenide ($WSe_2$).This model shows several significant improvements including: 1. When tested on simulated data, the model can recover the ground truth with minimal error. 2. The model can learn the rotation and strain on noisy diffraction patterns where CoM failed, and significantly outperforms template matching (py4DSTEM). 3. Our model can accommodate large and continuous rotations difficult to obtain with other methods. 4. Our model is more robust to noisy data. 5. Our model can map the strain, shear and rotation; identify dislocation and ripples; and distinguish background and sample area automatically.Ultimately, this work demonstrates how embedding physical concepts into unsupervised neural networks can simplify, automate, and accelerate analysis pipelines while simultaneously leveraging stochastic averaging that improves robustness on noisy data. This algorithmic concept can be extended to include other physical phenomena (e.g., polarization, sample tilt), can be used in automated experiments, and can be applied to other applications in materials characterization.Detailed information is attached in PDF.

In materials science, the challenge of rapid prototyping materials with desired properties often involves extensive experimentation to find suitable microstructures. Additionally, finding microstructures for given properties is typically an ill-posed problem where multiple solutions may exist. Using generative machine learning models can be a viable solution which also reduces the computational cost. This comes with new challenges because, e.g., a continuous property variable as conditioning input to the model is required. We investigate the shortcomings of an existing method and compare this to a novel embedding strategy for generative models that is based on the binary representation of floating point numbers. This eliminates the need for normalization, preserves information, and creates a versatile embedding space for conditioning the generative model. This technique can be applied to condition a network on any number, to provide fine control over generated microstructure images, thereby contributing to accelerated materials design.

Predicting properties from molecular structures is paramount to design tasks in medicine, materials science, and environmental management. However, design rules derived from the structure-property relationships using correlative data-driven methods fail to elucidate underlying causal mechanisms controlling chemical phenomena. This preliminary work proposes a workflow to actively learn robust cause-effect relations between structural features and molecular property for a broad chemical space utilizing smaller subsets, entailing partial information.

The rvesimulator aims to provide a user-friendly, automated Python-based framework conducting Representative Volume Element (RVE) simulation via powerful Finite Element Method (FEM) software Abaqus. By utilizing this repository, large amount of reliable FEM data-set generation is possible with RVEs encompassing materials from elastic to plastic composites. rvesimulator provides: (1) a cross-platform function to run arbitrary Python-Abaqus script without graphical user interface (GUI), it offers users a convenience way to run their unique scripts; (2) Python-Abaqus scripts to simulate RVE with different design of experiments including various micro-structures, material laws, and loading; (3) benchmarks of running prevalent RVEs covering elastic, hyper-elastic, plastic materials are provided, which illustrates the general pipeline (preprocess, execution, and postprocess) of the developed framework. By sharing the developing framework, we are aiming to reduce the labor-intensive process of generating massive of simulations data for new materials and structure discovery. Therefore, it facilitates the application and development of machine learning method for new material discovery.

The significance of Nuclear Magnetic Resonance (NMR) spectroscopy in organic synthesis cannot be overstated, as it plays a pivotal role in deducing chemical structures from experimental data. While machine learning has predominantly been employed for predictive purposes in the analysis of spectral data, our study introduces a novel application of a transformer-based model's attention weights to unravel the underlying "language" that correlates spectral peaks with their corresponding atom in the chemical structures. This attention mapping technique proves beneficial for comprehending spectra, enabling the reliable differentiation between product $^1$H-NMR spectra and reactant spectra extracted from experimental data with an accuracy exceeding 95\%. Furthermore, it consistently associates peaks with the correct atoms in the molecule, achieving a remarkable peak-to-atom match rate of 71\% for exact match and 89\% of close shift matching ($\pm$ 0.59ppm).This framework exemplifies the capability of harnessing the attention mechanism within transformer models to unveil the intricacies of spectroscopic data. Importantly, this approach can readily be extended to other types of spectra, showcasing its versatility and potential for broader applications in the field.

Deep reinforcement learning has successfully been applied for molecular discovery as shown by the Molecule Deep Q-network (MolDQN) algorithm. This algorithm has challenges when applied to optimizing new molecules: training such a model is limited in terms of scalability to larger datasets and the trained model cannot be generalized to different molecules in the same dataset.In this paper, a distributed reinforcement learning algorithm for antioxidants, called DA-MolDQN is proposed to address these problems.State-of-the-art bond dissociation energy (BDE) and ionization potential (IP) predictors are integrated into DA-MolDQN, which are critical chemical properties while optimizing antioxidants.Training time is reduced by algorithmic improvements for molecular modifications. The algorithm is distributed, scalable for up to 512 molecules, and generalizes the model to a diverse set of molecules.The proposed models are trained with a proprietary antioxidant dataset. The results have been reproduced with both proprietary and public datasets. The proposed molecules have been validated with DFT simulations and a subset of them confirmed in public "unseen" datasets.In summary, DA-MolDQN is up to 100x faster than previous algorithms and can discover new optimized molecules from proprietary and public antioxidants.

Modern microelectronic devices are composed of interfaces between a large number of materials, many of which are in amorphous or polycrystalline phases. Modeling such non-crystalline materials using first-principles methods such as density functional theory is often numerically intractable. Recently, graph neural networks (GNNs) have shown potential to achieve linear complexity with accuracies comparable to ab-initio methods. Here, we demonstrate the applicability of GNNs to accelerate the atomistic computational pipeline for predicting macroscopic transistor transport characteristics via learning microscopic physical properties. We generate amorphous heterostructures, specifically the HfO$_{2}$-SiO$_{2}$-Si semiconductor-dielectric transistor gate stack, via GNN predicted atomic forces, and show excellent accuracy in predicting transport characteristics including injection velocity for nanoslab silicon channels. This work paves the way for faster and more scalable methods to model modern advanced electronic devices via GNNs.

Sodium-ion batteries (SIBs) have been gaining attention for applications like grid-scale energy storage, largely owing to the abundance of sodium and an expected favorable $/kWh figure. Improving the performance of SIB electrode materials will enable these batteries to compete with mature technologies like lithium-ion batteries (LIBs) at scale. SIBs can leverage the well-established manufacturing knowledge of LIBs, but several materials synthesis and performance challenges for electrode materials need to be addressed for SIBs to mature to an industrial scale. This work extracts challenges in the performance and synthesis of SIB cathode active materials (CAMs) and reviews corresponding mitigation strategies from a combination of SIB and related LIB literature employing custom natural language processing (NLP) tools. These NLP tools help in identifying the mitigation strategies of interest and subsequently evaluate them using a process-based cost model and other scalability metrics. This approach facilitates the generation of quantitative insights and enables a unique comparison among a broad set of lab-proposed mitigation strategies. These derived insights enable engineers in research and industry to navigate a large number of proposed strategies and focus on impactful scalability-informed mitigation strategies to accelerate the transition from lab to commercialization.

A major challenge in DNA nanotechnology is to extend the length scale of DNA structures from the nanoscale to the microscale for biomedical, sensing, optical, and soft robotics applications. Self-assembly of DNA origami building blocks provides a promising approach for fabricating such higher-order structures. Inspired by self-assembly of patchy colloidal particles, researchers have recently begun to introduce patches of mutually attractive moieties, including non-natural hydrophobic polynucleotide brushes, at designated sites on DNA origami to assemble them into complex higher-order architectures. However, the underlying relationship between the design of these DNA origami building blocks and the resulting assembly structure is complex. Machine learning is especially well suited for such inverse-design tasks. In this work, we developed a coarse-grained model of DNA origami nanocubes grafted with hydrophobic brushes and employed neural adjoint (NA) method to explore highly ordered target assemblies, including checkerboard, honeycomb, and Kagome lattices. We envision that such inverse design approaches can be generalized to more complex designs and used to tailor structural properties to expand the application space of DNA nanotechnology.

Continuous dynamical systems, characterized by differential equations, are ubiquitously used to model several important problems: plasma dynamics, flow through porous media, weather forecasting, and epidemic dynamics. Recently, a wide range of data-driven models has been used successfully to model these systems. However, in contrast to established fields like computer vision, limited studies are available analyzing the strengths and potential applications of different classes of these models that could steer decision-making in scientific machine learning. Here, we introduce CoDBENCH, an exhaustive benchmarking suite comprising 11 state-of-the-art data-driven models for solving differential equations. Specifically, we comprehensively evaluate 4 distinct categories of models, viz., feed forward neural networks, deep operator regression models, frequency-based neural operators, and transformer architectures against 8 widely applicable benchmark datasets encompassing challenges from fluid and solid mechanics. We conduct extensive experiments, assessing the operators’ capabilities in learning, zero-shot super-resolution, data efficiency, robustness to noise, and computational efficiency. Interestingly, our findings highlight that current operators struggle with the newer mechanics datasets, motivating the need for more robust neural operators. All the datasets and codes are shared in an easy-to-use fashion for the scientific community. We hope this resource will be an impetus for accelerated progress and exploration in modeling dynamical systems. For codes and datasets, see: https://anonymous.4open.science/r/cod-bench-7525.

Neural networks (NNs) that exploit strong inductive biases based on physical laws and symmetries have shown remarkable success in learning the dynamics of physical systems directly from their trajectory. However, these works focus only on the systems that follow deterministic dynamics, such as Newtonian or Hamiltonian. Here, we propose a framework, namely Brownian graph neural networks (BroGNet), combining stochastic differential equations (SDEs) and GNNs to learn Brownian dynamics directly from the trajectory. We modify the architecture of BroGNet to enforce linear momentum conservation of the system, which, in turn, provides superior performance on learning dynamics as revealed empirically. We demonstrate this approach on several systems, namely, linear spring, linear spring with binary particle types, and non-linear spring systems, all following Brownian dynamics at finite temperatures. We show that BroGNet significantly outperforms proposed baselines across all the benchmarked Brownian systems. In addition, we demonstrate zero-shot generalizability of BroGNet to simulate unseen system sizes that are two orders of magnitude larger and to different temperatures than those used during training. Finally, we show that BroGNet conserves the momentum of the system, resulting in superior performance and data efficiency. Altogether, our study contributes to advancing the understanding of the intricate dynamics of Brownian motion and demonstrates the effectiveness of graph neural networks in modeling such complex systems.