Deep Reinforcement Learning of Transition States

Combining reinforcement learning (RL) and molecular dynamics (MD) simulations, we propose a machine-learning approach, called RL‡, to automatically unravel chemical reaction mechanisms. In RL‡, locating the transition state of a chemical reaction is formulated as a game, and two functions are optimized, one for value estimation and the other for policy making, to iteratively improve our chance of winning this game. Both functions can be approximated by deep neural networks. By virtue of RL‡, one can directly interpret the reaction mechanism according to the value function. Meanwhile, the policy function allows efficient sampling of the transition path ensemble, which can be further used to analyze reaction dynamics and kinetics. Through multiple experiments, we show that RL‡ can be trained tabula rasa hence allowing us to reveal chemical reaction mechanisms with minimal subjective biases.