A Group Symmetric Stochastic Differential Equation Model for Molecule
  Multi-modal Pretraining
Shengchao Liu Weitao Du Zhiming Ma Hongyu Guo Jian Tang
Molecule pretraining has quickly become the go-to schema to boost the
performance of AI-based drug discovery. Naturally, molecules can be represented
as 2D topological graphs or 3D geometric point clouds. Although most existing
pertaining methods focus on merely the single modality, recent research has
shown that maximizing the mutual information (MI) between such two modalities
enhances the molecule representation ability. Meanwhile, existing molecule
multi-modal pretraining approaches approximate MI based on the representation
space encoded from the topology and geometry, thus resulting in the loss of
critical structural information of molecules. To address this issue, we propose
MoleculeSDE. MoleculeSDE leverages group symmetric (e.g., SE(3)-equivariant and
reflection-antisymmetric) stochastic differential equation models to generate
the 3D geometries from 2D topologies, and vice versa, directly in the input
space. It not only obtains tighter MI bound but also enables prosperous
downstream tasks than the previous work. By comparing with 17 pretraining
baselines, we empirically verify that MoleculeSDE can learn an expressive
representation with state-of-the-art performance on 26 out of 32 downstream
tasks.