Deep Convolutional Neural Networks with Zero-Padding: Feature Extraction
  and Learning
Zhi Han Baichen Liu Shao-Bo Lin Ding-Xuan Zhou
This paper studies the performance of deep convolutional neural networks
(DCNNs) with zero-padding in feature extraction and learning. After verifying
the roles of zero-padding in enabling translation-equivalence, and pooling in
its translation-invariance driven nature, we show that with similar number of
free parameters, any deep fully connected networks (DFCNs) can be represented
by DCNNs with zero-padding. This demonstrates that DCNNs with zero-padding is
essentially better than DFCNs in feature extraction. Consequently, we derive
universal consistency of DCNNs with zero-padding and show its
translation-invariance in the learning process. All our theoretical results are
verified by numerical experiments including both toy simulations and real-data
running.