GloNets: Globally Connected Neural Networks
Antonio Di Cecco Carlo Metta Marco Fantozzi Francesco Morandin Maurizio Parton
Deep learning architectures suffer from depth-related performance
degradation, limiting the effective depth of neural networks. Approaches like
ResNet are able to mitigate this, but they do not completely eliminate the
problem. We introduce Globally Connected Neural Networks (GloNet), a novel
architecture overcoming depth-related issues, designed to be superimposed on
any model, enhancing its depth without increasing complexity or reducing
performance. With GloNet, the network's head uniformly receives information
from all parts of the network, regardless of their level of abstraction. This
enables GloNet to self-regulate information flow during training, reducing the
influence of less effective deeper layers, and allowing for stable training
irrespective of network depth. This paper details GloNet's design, its
theoretical basis, and a comparison with existing similar architectures.
Experiments show GloNet's self-regulation ability and resilience to
depth-related learning challenges, like performance degradation. Our findings
suggest GloNet as a strong alternative to traditional architectures like
ResNets.