Deep Predictive Coding with Bi-directional Propagation for
  Classification and Reconstruction
Senhui Qiu Saugat Bhattacharyya Damien Coyle Shirin Dora
This paper presents a new learning algorithm, termed Deep Bi-directional
Predictive Coding (DBPC) that allows developing networks to simultaneously
perform classification and reconstruction tasks using the same weights.
Predictive Coding (PC) has emerged as a prominent theory underlying information
processing in the brain. The general concept for learning in PC is that each
layer learns to predict the activities of neurons in the previous layer which
enables local computation of error and in-parallel learning across layers. In
this paper, we extend existing PC approaches by developing a network which
supports both feedforward and feedback propagation of information. Each layer
in the networks trained using DBPC learn to predict the activities of neurons
in the previous and next layer which allows the network to simultaneously
perform classification and reconstruction tasks using feedforward and feedback
propagation, respectively. DBPC also relies on locally available information
for learning, thus enabling in-parallel learning across all layers in the
network. The proposed approach has been developed for training both, fully
connected networks and convolutional neural networks. The performance of DBPC
has been evaluated on both, classification and reconstruction tasks using the
MNIST and FashionMNIST datasets. The classification and the reconstruction
performance of networks trained using DBPC is similar to other approaches used
for comparison but DBPC uses a significantly smaller network. Further, the
significant benefit of DBPC is its ability to achieve this performance using
locally available information and in-parallel learning mechanisms which results
in an efficient training protocol. This results clearly indicate that DBPC is a
much more efficient approach for developing networks that can simultaneously
perform both classification and reconstruction.