On the Difficulty of Defending Contrastive Learning against Backdoor
  Attacks
Changjiang Li Ren Pang Bochuan Cao Zhaohan Xi Jinghui Chen Shouling Ji Ting Wang
Recent studies have shown that contrastive learning, like supervised
learning, is highly vulnerable to backdoor attacks wherein malicious functions
are injected into target models, only to be activated by specific triggers.
However, thus far it remains under-explored how contrastive backdoor attacks
fundamentally differ from their supervised counterparts, which impedes the
development of effective defenses against the emerging threat.
  This work represents a solid step toward answering this critical question.
Specifically, we define TRL, a unified framework that encompasses both
supervised and contrastive backdoor attacks. Through the lens of TRL, we
uncover that the two types of attacks operate through distinctive mechanisms:
in supervised attacks, the learning of benign and backdoor tasks tends to occur
independently, while in contrastive attacks, the two tasks are deeply
intertwined both in their representations and throughout their learning
processes. This distinction leads to the disparate learning dynamics and
feature distributions of supervised and contrastive attacks. More importantly,
we reveal that the specificities of contrastive backdoor attacks entail
important implications from a defense perspective: existing defenses for
supervised attacks are often inadequate and not easily retrofitted to
contrastive attacks. We also explore several alternative defenses and discuss
their potential challenges. Our findings highlight the need for defenses
tailored to the specificities of contrastive backdoor attacks, pointing to
promising directions for future research.