Performance of neural networks can be significantly improved by encoding known invariance for particular tasks. Many image classification tasks, such as those related to cellular imaging, exhibit invariance to rotation. In particular, to aid convolutional neural networks in learning rotation invariance, we consider a simple, efficient conic convolutional scheme that encodes rotational equivariance, along with a method for integrating the magnitude response of the 2D-discrete-Fourier transform (2D-DFT) to encode global rotational invariance. We call our new method the Conic Convolution and DFT Network (CFNet). We evaluated the efficacy of CFNet as compared to a standard CNN and group-equivariant CNN (G-CNN) for several different image classification tasks and demonstrated improved performance, including classification accuracy, computational efficiency, and its robustness to hyperparameter selection. Taken together, we believe CFNet represents a new scheme that has the potential to improve many imaging analysis applications. Though the appeal of neural networks is their versatility for arbitrary classification tasks, there is still much benefit in designing them for particular problem settings. In particular, their effectiveness can be greatly increased by encoding invariance to uniformative augmentations of the data BID17 . If such invariance is not explicitly encoded, the network must learn it from the data, perhaps with the help of data augmentation, requiring more parameters and thereby increasing its susceptibility to overfitting.A key invariance inherent to several computer vision settings, including satellite imagery and all forms of microscopy imagery, is rotation BID3 BID1 . Recently, there have been a variety of proposed approaches for encoding rotation equivariance and invariance, the most promising of which have formulated convolution over groups BID6 BID24 . Notably, G-CNNs have been applied to several biological imaging tasks, producing state-of-the-art results BID24 BID0 BID18 .Here we propose a new rotation-equivariant convolutional scheme, called conic convolution, which, in contrast to group convolution, encodes equivariance while still operating over only the spatial domain. Rather than convolving each filter across the entire image, as in standard convolution, rotated filters are convolved over corresponding conic regions of the input feature map that emanate from the origin, thereby transforming rotations in the input directly to rotations in the output. This scheme is intuitive, simple to implement, and computationally efficient. We also show that the method yields improved performance over group convolution on several relevant applications.Additionally, we propose the integration of the magnitude response of the 2D-discrete-Fourier transform (2D-DFT) into a transition layer between convolutional and fully-connected layers to encode rotational invariance. Though the insight of using the DFT to encode rotational invariance has been employed for texture classification using wavelets BID9 BID13 BID21 BID2 and for general image classification BID23 , as of yet, its application to CNNs has been overlooked. As in these prior works, rotations of the input are transformed to circular shifts, to which the magnitude response of the 2D-DFT is invariant, in the transformed space. Most other recently proposed rotationinvariance CNNs impose this invariance by applying a permutation-invariant operation, such as the average or maximum, over the rotation group, but since this operation is applied for each filter individually, possibly valuable pose information between filters is lost. In contrast, the 2D-DFT is able to integrate mutual pose information between different filter responses, yielding richer features for subsequent layers.We demonstrate the effectiveness of these two novel contributions for various applications: classifying rotated MNIST images, classifying synthetic images that model biomarker expression in microscopy images of cells, and localizing proteins in budding yeast cells BID15 . We show that CFNet improves classification accuracy generally over the standard raster convolution formulation and over the equivariant method of G-CNN across these settings. We also show that the 2D-DFT clearly improves performance across these diverse data sets, and that not only for conic convolution, but also for group convolution. Source code for the implementation of CFNet will be made available on GitHub.2 RELATED WORK BID6 introduced G-CNNs by formulating convolution over groups, including rotation, translation, and flips, for neural networks, which has inspired many subsequent improvements. By convolving over groups, equivariance to these groups is maintained throughout the convolutional layers, and invariance is enforced at the end of the network by pooling over groups. This work was improved upon by the design of steerable filters BID24 for convolution, similar to those proposed by BID25 , which allow for finer sampling of rotations of filters without inducing artifacts. Steerable filters were first proposed by BID10 and had been explored previously for image classification BID19 , but as shallow features in the context of HOG descriptors.An alternative means of encoding rotational equivariance is to transform the domain of the image to an alternative domain, such as the log-polar domain BID23 BID11 in which rotation becomes some other transformation that is easier to manage, in this case, translations. The suitability of this transformation depends upon the signal of interest, since this warping will introduce distortion, as pixels near the center of the image are sampled more densely than pixels near the perimeter. In addition, its stability to translations in the original domain is of concern. Our proposed CFNet, by convolving over conic regions, also encodes global rotation equivariance about the origin, but without introducing such distortion, which greatly helps mitigate its susceptibility to translation. The recently developed spatial transform layer BID12 and deformable convolutional layer BID7 allow the network to learn non-regular sampling patterns and can potentially help learning rotation invariance, though invariance is not explicitly enforced, which would most likely be a challenge for tasks with small training data.A simple means for achieving rotation equivariance and invariance was proposed by BID8 , in which feature maps of standard CNNs are made equivariant or invariant to rotation by combinations of cyclic slicing, stacking, rolling, and pooling. RotEqNet BID20 improved upon this idea by storing , for each feature map for a corresponding filter, only the maximal response across rotations and the value of the corresponding rotation, to preserve pose information. This approach yielded improved results and considerable storage savings over BID8 and G-CNN. These methods are most similar to our proposed conic convolution . However, in contrast, our method applies each filter only at the appropriate rotation within each conic region, which further saves on storage.To enforce rotation invariance, as noted, most of the previous methods apply some permutationinvariant, or pooling, operation over rotations. BID3 recently proposed a strategy of encouraging a network to learn a rotation invariant transform, and follow-up work improved this learning process by incorporating a Fisher discriminant penalty BID4 . However, the convolutional layers of the network do not maintain the property of rotation equivariance with the input image, which requires that the network learn this equivariance and could therefore hinder performance. Also, learning such a transform that generalizes to unseen data could prove difficult for settings with limited training data. BID23 previously proposed the 2D-DFT for rotational invariance. However , no method has yet been proposed to integrate the 2D-DFT into a rotation-equivariant CNN.
We propose conic convolution and the 2D-DFT to encode rotation equivariance into an neural network. In the context of image classification, the paper proposes a convolutional neural network architecture with rotation-equivariant feature maps that are eventually made rotation-invariant by using the magnitude of the 2D discrete Fourier transform (DFT). Authors provide a rotation invariant neural network via combining conic convolution and 2D-DFT