We present an efficient convolution kernel for Convolutional Neural Networks (CNNs) on unstructured grids using parameterized differential operators while focusing on spherical signals such as panorama images or planetary signals. 
 To this end, we replace conventional convolution kernels with linear combinations of differential operators that are weighted by learnable parameters. Differential operators can be efficiently estimated on unstructured grids using one-ring neighbors, and learnable parameters can be optimized through standard back-propagation. As a result, we obtain extremely efficient neural networks that match or outperform state-of-the-art network architectures in terms of performance but with a significantly lower number of network parameters. We evaluate our algorithm in an extensive series of experiments on a variety of computer vision and climate science tasks, including shape classification, climate pattern segmentation, and omnidirectional image semantic segmentation. Overall, we present (1) a novel CNN approach on unstructured grids using parameterized differential operators for spherical signals, and (2) we show that our unique kernel parameterization allows our model to achieve the same or higher accuracy with significantly fewer network parameters. A wide range of machine learning problems in computer vision and related areas require processing signals in the spherical domain; for instance, omnidirectional RGBD images from commercially available panorama cameras, such as Matterport , panaramic videos coupled with LIDAR scans from self-driving cars BID17 , or planetary signals in scientific domains such as climate science BID31 . Unfortunately, naively mapping spherical signals to planar domains results in undesirable distortions. Specifically, projection artifacts near polar regions and handling of boundaries makes learning with 2D convolutional neural networks (CNNs) particularly challenging and inefficient. Very recent work, such as BID10 and BID16 , propose network architectures that operate natively in the spherical domain, and are invariant to rotations in the SO(3) group. Such invariances are desirable in a set of problems -e.g., machine learning problems of molecules -where gravitational effects are negligible and orientation is arbitrary. However, for other different classes of problems at large, assumed orientation information is crucial to the predictive capability of the network. A good example of such problems is the MNIST digit recognition problem, where orientation plays an important role in distinguishing digits "6" and "9". Other examples include omnidirectional images, where images are naturally oriented by gravity; and planetary signals, where planets are naturally oriented by their axis of rotation.In this work, we present a new convolution kernel for CNNs on arbitrary manifolds and topologies, discretized by an unstructured grid (i.e., mesh), and focus on its applications in the spherical domain approximated by an icosahedral spherical mesh. We propose and evaluate the use of a new parameterization scheme for CNN convolution kernels, which we call Parameterized Differential Operators (PDOs), which is easy to implement on unstructured grids. We call the resulting convolution operator that operates on the mesh using such kernels the MeshConv operator. This parameterization scheme utilizes only 4 parameters for each kernel, and achieves significantly better performance Illustration for the MeshConv operator using parameterized differential operators to replace conventional learnable convolutional kernels. Similar to classic convolution kernels that establish patterns between neighboring values, differential operators computes "differences", and a linear combination of differential operators establishes similar patterns.than competing methods, with much fewer parameters. In particular, we illustrate its use in various machine learning problems in computer vision and climate science.In summary, our contributions are as follows:• We present a general approach for orientable CNNs on unstructured grids using parameterized differential operators.• We show that our spherical model achieves significantly higher parameter efficiency compared to state-of-the-art network architectures for 3D classification tasks and spherical image semantic segmentation.• We release and open-source the codes developed and used in this study for other potential extended applications 1 .We organize the structure of the paper as follows. We first provide an overview of related studies in the literature in Sec. 2; we then introduce details of our methodology in Sec. 3, followed by an empirical assessment of the effectiveness of our model in Sec. 4. Finally , we evaluate the design choices of our kernel parameterization scheme in Sec. 5. We have presented a novel method for performing convolution on unstructured grids using parameterized differential operators as convolution kernels. Our results demonstrate its applicability to machine learning problems with spherical signals and show significant improvements in terms of overall performance and parameter efficiency. We believe that these advances are particularly valuable with the increasing relevance of omnidirectional signals, for instance, as captured by real-world 3D or LIDAR panorama sensors.
We present a new CNN kernel for unstructured grids for spherical signals, and show significant accuracy and parameter efficiency gain on tasks such as 3D classfication and omnidirectional image segmentation. An efficient method enabling deep learning on spherical data that reaches competitive/state-of-the-art numbers with much less parameters than popular approaches. The paper proposes a novel convolutional kernel for CNN on the unstructured grids and formulates the convolution by a linear combination of differential operators.