Classification and Reconstruction Processes in Deep Predictive Coding
  Networks: Antagonists or Allies?
Jan Rathjens Laurenz Wiskott
Predictive coding-inspired deep networks for visual computing integrate
classification and reconstruction processes in shared intermediate layers.
Although synergy between these processes is commonly assumed, it has yet to be
convincingly demonstrated. In this study, we take a critical look at how
classifying and reconstructing interact in deep learning architectures. Our
approach utilizes a purposefully designed family of model architectures
reminiscent of autoencoders, each equipped with an encoder, a decoder, and a
classification head featuring varying modules and complexities. We meticulously
analyze the extent to which classification- and reconstruction-driven
information can seamlessly coexist within the shared latent layer of the model
architectures. Our findings underscore a significant challenge:
Classification-driven information diminishes reconstruction-driven information
in intermediate layers' shared representations and vice versa. While expanding
the shared representation's dimensions or increasing the network's complexity
can alleviate this trade-off effect, our results challenge prevailing
assumptions in predictive coding and offer guidance for future iterations of
predictive coding concepts in deep networks.