Convex Geometry of ReLU-layers, Injectivity on the Ball and Local
  Reconstruction
Daniel Haider Martin Ehler Peter Balazs
The paper uses a frame-theoretic setting to study the injectivity of a
ReLU-layer on the closed ball of $\mathbb{R}^n$ and its non-negative part. In
particular, the interplay between the radius of the ball and the bias vector is
emphasized. Together with a perspective from convex geometry, this leads to a
computationally feasible method of verifying the injectivity of a ReLU-layer
under reasonable restrictions in terms of an upper bound of the bias vector.
Explicit reconstruction formulas are provided, inspired by the duality concept
from frame theory. All this gives rise to the possibility of quantifying the
invertibility of a ReLU-layer and a concrete reconstruction algorithm for any
input vector on the ball.