Augmented Memory: Capitalizing on Experience Replay to Accelerate De
  Novo Molecular Design
Jeff Guo Philippe Schwaller
Sample efficiency is a fundamental challenge in de novo molecular design.
Ideally, molecular generative models should learn to satisfy a desired
objective under minimal oracle evaluations (computational prediction or wet-lab
experiment). This problem becomes more apparent when using oracles that can
provide increased predictive accuracy but impose a significant cost.
Consequently, these oracles cannot be directly optimized under a practical
budget. Molecular generative models have shown remarkable sample efficiency
when coupled with reinforcement learning, as demonstrated in the Practical
Molecular Optimization (PMO) benchmark. Here, we propose a novel algorithm
called Augmented Memory that combines data augmentation with experience replay.
We show that scores obtained from oracle calls can be reused to update the
model multiple times. We compare Augmented Memory to previously proposed
algorithms and show significantly enhanced sample efficiency in an exploitation
task and a drug discovery case study requiring both exploration and
exploitation. Our method achieves a new state-of-the-art in the PMO benchmark
which enforces a computational budget, outperforming the previous best
performing method on 19/23 tasks.