Convolutional Channel-wise Competitive Learning for the Forward-Forward
  Algorithm
Andreas Papachristodoulou Christos Kyrkou Stelios Timotheou Theocharis Theocharides
The Forward-Forward (FF) Algorithm has been recently proposed to alleviate
the issues of backpropagation (BP) commonly used to train deep neural networks.
However, its current formulation exhibits limitations such as the generation of
negative data, slower convergence, and inadequate performance on complex tasks.
In this paper, we take the main ideas of FF and improve them by leveraging
channel-wise competitive learning in the context of convolutional neural
networks for image classification tasks. A layer-wise loss function is
introduced that promotes competitive learning and eliminates the need for
negative data construction. To enhance both the learning of compositional
features and feature space partitioning, a channel-wise feature separator and
extractor block is proposed that complements the competitive learning process.
Our method outperforms recent FF-based models on image classification tasks,
achieving testing errors of 0.58%, 7.69%, 21.89%, and 48.77% on MNIST,
Fashion-MNIST, CIFAR-10 and CIFAR-100 respectively. Our approach bridges the
performance gap between FF learning and BP methods, indicating the potential of
our proposed approach to learn useful representations in a layer-wise modular
fashion, enabling more efficient and flexible learning.