AdsorbRL: Deep Multi-Objective Reinforcement Learning for Inverse
  Catalysts Design
Romain Lacombe Lucas Hendren Khalid El-Awady
A central challenge of the clean energy transition is the development of
catalysts for low-emissions technologies. Recent advances in Machine Learning
for quantum chemistry drastically accelerate the computation of catalytic
activity descriptors such as adsorption energies. Here we introduce AdsorbRL, a
Deep Reinforcement Learning agent aiming to identify potential catalysts given
a multi-objective binding energy target, trained using offline learning on the
Open Catalyst 2020 and Materials Project data sets. We experiment with Deep
Q-Network agents to traverse the space of all ~160,000 possible unary, binary
and ternary compounds of 55 chemical elements, with very sparse rewards based
on adsorption energy known for only between 2,000 and 3,000 catalysts per
adsorbate. To constrain the actions space, we introduce Random Edge Traversal
and train a single-objective DQN agent on the known states subgraph, which we
find strengthens target binding energy by an average of 4.1 eV. We extend this
approach to multi-objective, goal-conditioned learning, and train a DQN agent
to identify materials with the highest (respectively lowest) adsorption
energies for multiple simultaneous target adsorbates. We experiment with
Objective Sub-Sampling, a novel training scheme aimed at encouraging
exploration in the multi-objective setup, and demonstrate simultaneous
adsorption energy improvement across all target adsorbates, by an average of
0.8 eV. Overall, our results suggest strong potential for Deep Reinforcement
Learning applied to the inverse catalysts design problem.