Atom-Motif Contrastive Transformer for Molecular Property Prediction
Wentao Yu Shuo Chen Chen Gong Gang Niu Masashi Sugiyama
Recently, Graph Transformer (GT) models have been widely used in the task of
Molecular Property Prediction (MPP) due to their high reliability in
characterizing the latent relationship among graph nodes (i.e., the atoms in a
molecule). However, most existing GT-based methods usually explore the basic
interactions between pairwise atoms, and thus they fail to consider the
important interactions among critical motifs (e.g., functional groups consisted
of several atoms) of molecules. As motifs in a molecule are significant
patterns that are of great importance for determining molecular properties
(e.g., toxicity and solubility), overlooking motif interactions inevitably
hinders the effectiveness of MPP. To address this issue, we propose a novel
Atom-Motif Contrastive Transformer (AMCT), which not only explores the
atom-level interactions but also considers the motif-level interactions. Since
the representations of atoms and motifs for a given molecule are actually two
different views of the same instance, they are naturally aligned to generate
the self-supervisory signals for model training. Meanwhile, the same motif can
exist in different molecules, and hence we also employ the contrastive loss to
maximize the representation agreement of identical motifs across different
molecules. Finally, in order to clearly identify the motifs that are critical
in deciding the properties of each molecule, we further construct a
property-aware attention mechanism into our learning framework. Our proposed
AMCT is extensively evaluated on seven popular benchmark datasets, and both
quantitative and qualitative results firmly demonstrate its effectiveness when
compared with the state-of-the-art methods.