Transferability of Convolutional Neural Networks in Stationary Learning
  Tasks
Damian Owerko Charilaos I. Kanatsoulis Jennifer Bondarchuk Donald J. Bucci Jr Alejandro Ribeiro
Recent advances in hardware and big data acquisition have accelerated the
development of deep learning techniques. For an extended period of time,
increasing the model complexity has led to performance improvements for various
tasks. However, this trend is becoming unsustainable and there is a need for
alternative, computationally lighter methods. In this paper, we introduce a
novel framework for efficient training of convolutional neural networks (CNNs)
for large-scale spatial problems. To accomplish this we investigate the
properties of CNNs for tasks where the underlying signals are stationary. We
show that a CNN trained on small windows of such signals achieves a nearly
performance on much larger windows without retraining. This claim is supported
by our theoretical analysis, which provides a bound on the performance
degradation. Additionally, we conduct thorough experimental analysis on two
tasks: multi-target tracking and mobile infrastructure on demand. Our results
show that the CNN is able to tackle problems with many hundreds of agents after
being trained with fewer than ten. Thus, CNN architectures provide solutions to
these problems at previously computationally intractable scales.