Exploring the Relationship: Transformative Adaptive Activation Functions
  in Comparison to Other Activation Functions
Vladimír Kunc
Neural networks are the state-of-the-art approach for many tasks and the
activation function is one of the main building blocks that allow such
performance. Recently, a novel transformative adaptive activation function
(TAAF) allowing for any vertical and horizontal translation and scaling was
proposed. This work sets the TAAF into the context of other activation
functions. It shows that the TAAFs generalize over 50 existing activation
functions and utilize similar concepts as over 70 other activation functions,
underscoring the versatility of TAAFs. This comprehensive exploration positions
TAAFs as a promising and adaptable addition to neural networks.