Large-scale Pretraining Improves Sample Efficiency of Active Learning
  based Molecule Virtual Screening
Zhonglin Cao Simone Sciabola Ye Wang
Virtual screening of large compound libraries to identify potential hit
candidates is one of the earliest steps in drug discovery. As the size of
commercially available compound collections grows exponentially to the scale of
billions, brute-force virtual screening using traditional tools such as docking
becomes infeasible in terms of time and computational resources. Active
learning and Bayesian optimization has recently been proven as effective
methods of narrowing down the search space. An essential component in those
methods is a surrogate machine learning model that is trained with a small
subset of the library to predict the desired properties of compounds. Accurate
model can achieve high sample efficiency by finding the most promising
compounds with only a fraction of the whole library being virtually screened.
In this study, we examined the performance of pretrained transformer-based
language model and graph neural network in Bayesian optimization active
learning framework. The best pretrained models identifies 58.97% of the
top-50000 by docking score after screening only 0.6% of an ultra-large library
containing 99.5 million compounds, improving 8% over previous state-of-the-art
baseline. Through extensive benchmarks, we show that the superior performance
of pretrained models persists in both structure-based and ligand-based drug
discovery. Such model can serve as a boost to the accuracy and sample
efficiency of active learning based molecule virtual screening.