LISBET: a self-supervised Transformer model for the automatic
  segmentation of social behavior motifs
Giuseppe Chindemi Benoit Girard Camilla Bellone
Social behavior, defined as the process by which individuals act and react in
response to others, is crucial for the function of societies and holds profound
implications for mental health. To fully grasp the intricacies of social
behavior and identify potential therapeutic targets for addressing social
deficits, it is essential to understand its core principles. Although machine
learning algorithms have made it easier to study specific aspects of complex
behavior, current methodologies tend to focus primarily on single-animal
behavior. In this study, we introduce LISBET (seLf-supervIsed Social BEhavioral
Transformer), a model designed to detect and segment social interactions. Our
model eliminates the need for feature selection and extensive human annotation
by using self-supervised learning to detect and quantify social behaviors from
dynamic body parts tracking data. LISBET can be used in hypothesis-driven mode
to automate behavior classification using supervised finetuning, and in
discovery-driven mode to segment social behavior motifs using unsupervised
learning. We found that motifs recognized using the discovery-driven approach
not only closely match the human annotations but also correlate with the
electrophysiological activity of dopaminergic neurons in the Ventral Tegmental
Area (VTA). We hope LISBET will help the community improve our understanding of
social behaviors and their neural underpinnings.