This paper introduces the concept of continuous convolution to neural networks and deep learning applications in general. Rather than directly using discretized information, input data is first projected into a high-dimensional Reproducing Kernel Hilbert Space (RKHS), where it can be modeled as a continuous function using a series of kernel bases. We then proceed to derive a closed-form solution to the continuous convolution operation between two arbitrary functions operating in different RKHS. Within this framework, convolutional filters also take the form of continuous functions, and the training procedure involves learning the RKHS to which each of these filters is projected, alongside their weight parameters. This results in much more expressive filters, that do not require spatial discretization and benefit from properties such as adaptive support and non-stationarity. Experiments on image classification are performed, using classical datasets, with results indicating that the proposed continuous convolutional neural network is able to achieve competitive accuracy rates with far fewer parameters and a faster convergence rate. In recent years, convolutional neural networks (CNNs) have become widely popular as a deep learning tool for addressing various sorts of problems, most predominantly in computer vision, such as image classification BID22 BID17 , object detection BID30 and semantic segmentation BID8 . The introduction of convolutional filters produces many desirable effects, including: translational invariance, since the same patterns are detected in the entire image; spatial connectivity, as neighboring information is taken into consideration during the convolution process; and shared weights, which results in significantly fewer training parameters and smaller memory footprint.Even though the convolution operation is continuous in nature BID19 , a common assumption in most computational tasks is data discretization, since that is usually how information is obtained and stored: 2D images are divided into pixels, 3D point clouds are divided into voxels, and so forth. Because of that, the exact convolution formulation is often substituted by a discrete approximation BID4 , calculated by sliding the filter over the input data and calculating the dot product of overlapping areas. While much simpler to compute, it requires substantially more computational power, especially for larger datasets and filter sizes BID28 . The fast Fourier transform has been shown to significantly increase performance in convolutional neural network calculations BID18 BID32 , however these improvements are mostly circumstantial, with the added cost of performing such transforms, and do not address memory requirements.To the best of our knowledge, all versions of CNNs currently available in the literature use this discrete approximation to convolution, as a way to simplify calculations at the expense of a potentially more descriptive model. In BID24 a sparse network was used to dramatically decrease computational times by exploiting redundancies, and in BID11 spatial sparseness was exploited to achieve state-of-the-art results in various image classification datasets. Similarly, BID31 used octrees to efficiently partition the space during convolution, thus focusing memory allocation and computation to denser regions. A quantized version was proposed in BID40 to improve performance on mobile devices, with simultaneous computational acceleration and model compression. A lookup-based network is described in BID0 , that encodes convolution as a series of lookups to a dictionary that is trained to cover the observed weight space. This paper takes a different approach and introduces the concept of continuous convolution to neural networks and deep learning applications in general. This is achieved by projecting information into a Reproducing Kernel Hilbert Space (RKHS) BID33 , in which point evaluation takes the form of a continuous linear functional. We employ the Hilbert Maps framework, initially described in BID29 , to reconstruct discrete input data as a continuous function, based on the methodology proposed in BID12 . Within this framework, we derive a closed-form solution to the continuous convolution between two functions that takes place directly in this high-dimensional space, where arbitrarily complex patterns are represented using a series of simple kernels, that can be efficiently convolved to produce a third RKHS modeling activation values. Optimizing this neural network involves learning not only weight parameters, but also the RKHS that defines each convolutional filter, which results is much more descriptive feature maps that can be used for both discriminative and generative tasks. The use of high-dimensional projection, including infinite-layer neural networks BID15 ; BID9 , has been extensively studied in recent times, as a way to combine kernel-based learning with deep learning applications. Note that, while works such as BID26 and BID25 take a similar approach of projecting input data into a RKHS, using the kernel trick, it still relies on discretized image patches, whereas ours operates solely on data already projected to these highdimensional spaces. Also, in these works extra kernel parameters are predetermined and remain fixed during the training process, while ours jointly learns these parameters alongside traditional weight values, thus increasing the degrees of freedom in the resulting feature maps.The proposed technique, entitled Continuous Convolutional Neural Networks (CCNNs), was evaluated in an image classification context, using standard computer vision benchmarks, and achieved competitive accuracy results with substantially smaller network sizes. We also demonstrate its applicability to unsupervised learning, by describing a convolutional auto-encoder that is able to produce latent feature representations in the form of continuous functions, which are then used as initial filters for classification using labeled data. This paper introduced a novel technique for data representation that takes place in a highdimensional Reproducing Kernel Hilbert Space (RKHS), where arbitrarily complex functions can be approximated in a continuous fashion using a series of simple kernels. We show how these kernels can be efficiently convolved to produce approximations of convolution results between two functions in different RKHS, and how this can be applied in an image classification scenario, via the introduction of a novel deep learning architecture entitled Continuous Convolutional Neural Networks (CCNN). Experimental tests using standard benchmark datasets show that this proposed architecture is able to achieve competitive results with much smaller network sizes, by focusing instead on more descriptive individual filters that are used to extract more complex patterns. Although promising, there are still several potential improvements that are left for future work, such as: RKHS sparsification, so only a subset of clusters are used for feature vector calculation, which would greatly improve computational speed and memory requirements; different learning rates and optimization strategies for each class of parameter (cluster location, length-scale and weight), to improve convergence rates; and the use of different kernels for feature vector representation, as a way to encode different properties in the resulting feature maps.
This paper proposes a novel convolutional layer that operates in a continuous Reproducing Kernel Hilbert Space. Projecting examples into an RK Hilbert space and performing convolution and filtering into that space. This paper formulates a variant of convolutional neural networks which models both activations and filters as continuous functions composed from kernel bases