Artificial neural networks are built on the basic operation of linear combination and non-linear activation function. Theoretically this structure can approximate any continuous function with three layer architecture. But in practice learning  the parameters of such network can be hard. Also the choice of activation function can greatly impact the performance of the network. In this paper we are proposing to replace the basic linear combination operation with non-linear operations that do away with the need of additional non-linear activation function. To this end we are proposing the use of elementary  morphological operations (dilation and erosion) as the basic operation in neurons. We show that these networks (Denoted as Morph-Net) with morphological operations can approximate any smooth function requiring less number of parameters than what is necessary for normal neural networks. The results show that our network perform favorably when compared with similar structured network. We have carried out our experiments on  MNIST, Fashion-MNIST, CIFAR10 and CIFAR100. In artificial neural networks, the basic building block is an artificial neuron or perceptron that simply computes the linear combination of the input BID22 . It is usually followed by a non-linear activation function to model the non-linearity of the output. Although the neurons are simple in nature, when connected together they can approximate any continuous function of the input BID4 . This has been successfully utilized in solving different real world problems like image classification (Krizhevsky et al., 2012) , semantic segmentation BID13 and image generation BID6 . While these models are quite powerful in nature, their efficient training can be hard in general BID12 and they need support of specials techniques, such as batch normalization BID5 and dropout BID24 , in order to achieve better generalization capabilities. Their training time also depends on the choice of activation function BID15 .In this paper we are proposing new building blocks for building networks similar to neural network. Here , instead of the linear combination operation of the artificial neurons, we use a non-linear operation that eliminates the need of additional activation function while requiring a small number of neurons to attain same performance or better. More specifically, We use morphological operations (i.e. dilation and erosion) as the elementary operation of the neurons in the network. Our contribution in this paper is building a network with these operations that has the following properties.1. Networks built with with dilation-erosion neurons followed by linear combination can approximate any continuous function given enough dilation/erosion neurons. 2. As dilation and erosion operation are non-linear by themselves, requirement of separate non-linear activation function is eliminated. 3. The use of dilation-erosion operation greatly increases number of possible decision boundaries. As a result, complex decision boundaries can be learned using small number of parameters.The rest of the paper is organized as follows. Section 2 describes the prior work on morphological neural network. In Section 3, we introduce our proposed network and prove its capabilities theoretically. We further demonstrate its capabilities empirically on a few benchmark datasets in Section 4. Lastly Section 6 concludes the paper. In this paper we have proposed a new class of networks that uses both normal and morphological neurons. These network consists of three layers only: input layer, dilation-erosion layer with dilation and erosion neurons followed by linear combination layer giving the output of the network with normal artificial neurons. We have done our analysis using this three layer network only, but its deeper version can also be explored. We have shown that this three layer architecture can approximate any sufficiently smooth function without requiring any non-linear activation function. These networks are able to learn a large number of hyperplanes with very few neurons in the dilation-erosion layer thereby providing superior results compared to other networks with three layer architecture. The improved results could also be the result of 'feature selection' by the max/min operator in the dilation erosion layer. In this work we have only worked with fully connected layers, i.e. a node in a layer is connected to all the nodes in the previous layer. This type of connectivity is not very efficient for image data where architectures with convolution layers perform better. So, extending this work to the case where a structuring element operates by sliding over the whole image, should be the next logical step.APPENDIX A PROOF OF LEMMA 1 From equation 9 we have DISPLAYFORM0 Now this equation can be rewritten as follows DISPLAYFORM1 where s + ik and s − ik denote the k th component of the i th structuring element of dilation and erosion neurons, respectively. The above equation can be further expressed in the following form, DISPLAYFORM2 Where DISPLAYFORM3 are define in the following way DISPLAYFORM4 Now, without any loss of generality we can write equation 17 as follows DISPLAYFORM5 where DISPLAYFORM6 Finally, we can rewrite equation 18 as DISPLAYFORM7 where l = m + n, α i ∈ {1, −1} and φ i (x)'s are of the following form DISPLAYFORM8 DISPLAYFORM9 In equation 20, v DISPLAYFORM10 represents sum of multi-oder hinge function.However, it may be noted that taking l ≥ d results hinge hyper planes which can span any where in d dimensional input space. We can assume there are l 1 and l 2 number of terms where α = 1 and α = −1 respectively, then DISPLAYFORM11 where l 1 + l 2 = l and φ i (x), φ i (x) is of same form as equation 20. Threfore can write, DISPLAYFORM12 DISPLAYFORM13 where k i ∈ {1, 2, .. , d + 1}∀i. In equation 24 we are taking maximum of (d + 1) l1 terms. Similarly we can derive same expression for Proof From equation 19, without any loss of generality we can assume there are t 1 and t 2 number of terms where α = 1 and α = −1 respectively, then DISPLAYFORM14 where t 1 + t 2 = l and φ i (x), φ i (x) are of same form as equation 20.As sum of PWL functions is also a PWL function, hence each t1 i=1 φ i (x) and t2 i=1 φ i (x) and PWL. Now, if t 1 > 0, from Proposition 3 we can conclude that g(x) is PWL linear function since difference of two continuous PWL function is PWL function . If t 1 = 0 then g(x) becomes PWL concave function.Hence, can say g(x) is PWL function.It may be noted that if l < d then PWL hyperplane will be in parallel to at least one of the axis. Taking l ≥ d results PWL hyperplane which may span anywhere in d dimensional space.Theorem 2 (Universal approximation) Using only a single dilation-erosion layer followed by a linear combination layer any continuous function can be approximated.
Using mophological operation (dilation and erosion) we have defined a class of network which can approximate any continious function.  This paper proposes to replace standard RELU/tanh units with a combination of dilation and erosion operations, observing that the new operator creates more hyper-planes and has more expressive power. The authors introduce Morph-Net, a single layer neural network where the mapping is performed using morphological dilation and erosion.