 In 1989, the first proof was published by George Cybenko for sigmoid activation functions and was generalised to feed-forward multi-layer architectures in 1991 by Kurt Hornik.Recent work also showed that universal approximation also holds for non-bounded activation functions such as the rectified linear unit.The universal approximation theorem for deep neural networks concerns the capacity of networks with bounded width but the depth is allowed to grow.