Mitigating Communication Costs in Neural Networks: The Role of Dendritic
  Nonlinearity
Xundong Wu Pengfei Zhao Zilin Yu Lei Ma Ka-Wa Yip Huajin Tang Gang Pan Tiejun Huang
Our comprehension of biological neuronal networks has profoundly influenced
the evolution of artificial neural networks (ANNs). However, the neurons
employed in ANNs exhibit remarkable deviations from their biological analogs,
mainly due to the absence of complex dendritic trees encompassing local
nonlinearity. Despite such disparities, previous investigations have
demonstrated that point neurons can functionally substitute dendritic neurons
in executing computational tasks. In this study, we scrutinized the importance
of nonlinear dendrites within neural networks. By employing machine-learning
methodologies, we assessed the impact of dendritic structure nonlinearity on
neural network performance. Our findings reveal that integrating dendritic
structures can substantially enhance model capacity and performance while
keeping signal communication costs effectively restrained. This investigation
offers pivotal insights that hold considerable implications for the development
of future neural network accelerators.