Global-based Deep Q-Network for Molecule Generation

In this work, we propose the Global-based Deep Q-Network for Molecule Generation (mol-GDQN). Explicitly, the proposed mol-GDQN, designs a novel molecular drug by iteratively adding or deleting atoms, bounds or chemical fragments from a lead molecule until reaching a molecular structure with all target properties. We formulate the drug design problem as a Markov Decision Process (MDP), where the RL-agent is defined as the molecule generator. Instead of defining the agent's observations using only local information, we pass a global observation of the molecule as input to the agent at each iteration. We define the global observations of the dynamically changing molecule using a novel GNN variant that is based on a global message passing schema. According to the Markov property, the global observations of the molecule make the agent's actions independent from the previous molecule states contrary to the local observations. Therefore, when receiving global observations of the molecule, the defined RL-agent, carries more optimal actions leading to better properties improvement compared to the local-based approaches. Using the proposed global-based GNN variant, we further define a global observation of the original lead molecule and pass it as additional input to the RL-agent. The obtained results showed that when the RL-agent receives a global observation of the lead molecule throughout the generation process, it enables it to preserve the properties of the lead molecules in the newly generated molecules.