WavPool: A New Block for Deep Neural Networks
Samuel D. McDermott M. Voetberg Brian Nord
Modern deep neural networks comprise many operational layers, such as dense
or convolutional layers, which are often collected into blocks. In this work,
we introduce a new, wavelet-transform-based network architecture that we call
the multi-resolution perceptron: by adding a pooling layer, we create a new
network block, the WavPool. The first step of the multi-resolution perceptron
is transforming the data into its multi-resolution decomposition form by
convolving the input data with filters of fixed coefficients but increasing
size. Following image processing techniques, we are able to make scale and
spatial information simultaneously accessible to the network without increasing
the size of the data vector. WavPool outperforms a similar multilayer
perceptron while using fewer parameters, and outperforms a comparable
convolutional neural network by ~ 10% on relative accuracy on CIFAR-10.