TSSR: A Truncated and Signed Square Root Activation Function for Neural
  Networks
Yuanhao Gong
Activation functions are essential components of neural networks. In this
paper, we introduce a new activation function called the Truncated and Signed
Square Root (TSSR) function. This function is distinctive because it is odd,
nonlinear, monotone and differentiable. Its gradient is continuous and always
positive. Thanks to these properties, it has the potential to improve the
numerical stability of neural networks. Several experiments confirm that the
proposed TSSR has better performance than other stat-of-the-art activation
functions. The proposed function has significant implications for the
development of neural network models and can be applied to a wide range of
applications in fields such as computer vision, natural language processing,
and speech recognition.