Using the Transformer Model for Physical Simulation: An application on Transient Thermal Analysis for 3D Printing Process Simulation

Transient thermal analysis is widely used in many science and engineering areas such as electronic packaging, engine design and manufacturing. High dimensional simulations are very expensive to run. Here we propose a machine learning model consisting of a pre-trained convolutional neural network (CNN), a transformer encoder and a multilayer perceptron (MLP) to predict the temperature field of 3D printed parts. The CAD part used in 3D printing is firstly sliced into layers and represented as images. We use the pre-trained ResNet 34 to extract low level geometry features, taking the output feature map of its Conv_4 layer as the geometry embedding vector. The transformer encoder are used to capture the long-range dependencies between layer-wise geometry features. MLP then takes the transformer's output and predicts the temperatures at given locations and time step. Our results show the model can accurately predict the thermal history in 3D printing process on different geometries. Our model is also very efficient, running 1~2 orders of magnitude faster than the simulation on which it is trained, without requiring the complicated pre-processing steps in transient thermal analysis including CAD file fix, material property setup, mesh generation and refinement, and defining the boundary conditions and dynamic loading in every time step.