A Synaptic Neural Network (SynaNN) consists of synapses and neurons. Inspired by the synapse research of neuroscience, we built a synapse model with a nonlinear synapse function of excitatory and inhibitory channel probabilities. Introduced the concept of surprisal space and constructed a commutative diagram, we proved that the inhibitory probability function -log(1-exp(-x)) in surprisal space is the topologically conjugate function of the inhibitory complementary probability 1-x in probability space. Furthermore, we found that the derivative of the synapse over the parameter in the surprisal space is equal to the negative Bose-Einstein distribution. In addition, we constructed a fully connected synapse graph (tensor) as a synapse block of a synaptic neural network. Moreover, we proved the gradient formula of a cross-entropy loss function over parameters, so synapse learning can work with the gradient descent and backpropagation algorithms. In the proof-of-concept experiment, we performed an MNIST training and testing on the MLP model with synapse network as hidden layers. Synapses play an important role in biological neural networks BID11 ). They are joint points of neurons' connection with the capability of learning and memory in neural networks. Based on the analysis of excitatory and inhibitory channels of synapses BID11 ), we proposed a probability model BID6 for probability introduction) of the synapse together with a non-linear function of excitatory and inhibitory probabilities BID17 (synapse function)). Inspired by the concept of surprisal from (Jones (1979)(self-information), BID15 , BID2 (surprisal analysis), BID16 (surprisal theory in language)) or negative logarithmic space BID21 ), we proposed the concept of surprisal space and represented the synapse function as the addition of the excitatory function and inhibitory function in the surprisal space. By applying a commutative diagram, we figured out the fine structure of inhibitory function and proved that it was the topologically conjugate function of an inhibitory function. Moreover, we discovered (rediscovered) that the derivative of the inhibitory function over parameter was equal to the negative Bose-Einstein distribution BID22 ). Furthermore, we constructed a fully connected synapse graph and figured out its synapse tensor expression. From synapse tensor and a cross-entropy loss function, we found and proved its gradient formula that was the basis for gradient descent learning and using backpropagation algorithm. In surprisal space, the parameter (weight) updating for learning was the addition of the value of the negative Bose-Einstein distribution. Finally, we designed the program to implement a Multiple Layer Perceptrons (MLP) BID20 ) for MNIST BID14 ) and tested it to achieve the near equal accuracy of standard MLP in the same setting. In this paper, we presented and analyzed a Synaptic Neural Network (SynaNN). We found the fine structure of synapse and the construction of synapse network as well as the BE distribution in the gradient descent learning. In surprisal space, the input of a neuron is the addition of the identity function and the sum of topologically conjugate functions of inhibitory synapses which is the sum of bits of information. The formula of surprisal synapse function is defined as LS(u, v; θ, γ) = (θ + u) + (I • F • I −1 )(γ + v))The non-linear synaptic neural network may be implemented by physical or chemical components.Instead of using a simple linear synapse function, more synapse functions maybe found in the researches and applications of neural network.
A synaptic neural network with synapse graph and learning that has the feature of topological conjugation and Bose-Einstein distribution in surprisal space.   The authors propose a hybrid neural nework composed of a synapse graph that can be embedded into a standard neural network Presents a biologically-inspired neural network model based on the excitatory and inhibitory ion channels in the membranes of real cells