Unlocking the Potential of Similarity Matching: Scalability, Supervision
  and Pre-training
Yanis Bahroun Shagesh Sridharan Atithi Acharya Dmitri B. Chklovskii Anirvan M. Sengupta
While effective, the backpropagation (BP) algorithm exhibits limitations in
terms of biological plausibility, computational cost, and suitability for
online learning. As a result, there has been a growing interest in developing
alternative biologically plausible learning approaches that rely on local
learning rules. This study focuses on the primarily unsupervised similarity
matching (SM) framework, which aligns with observed mechanisms in biological
systems and offers online, localized, and biologically plausible algorithms. i)
To scale SM to large datasets, we propose an implementation of Convolutional
Nonnegative SM using PyTorch. ii) We introduce a localized supervised SM
objective reminiscent of canonical correlation analysis, facilitating stacking
SM layers. iii) We leverage the PyTorch implementation for pre-training
architectures such as LeNet and compare the evaluation of features against
BP-trained models. This work combines biologically plausible algorithms with
computational efficiency opening multiple avenues for further explorations.