SkelEx and BoundEx: Natural Visualization of ReLU Neural Networks
Pawel Pukowski Haiping Lu
Despite their limited interpretability, weights and biases are still the most
popular encoding of the functions learned by ReLU Neural Networks (ReLU NNs).
That is why we introduce SkelEx, an algorithm to extract a skeleton of the
membership functions learned by ReLU NNs, making those functions easier to
interpret and analyze. To the best of our knowledge, this is the first work
that considers linear regions from the perspective of critical points. As a
natural follow-up, we also introduce BoundEx, which is the first analytical
method known to us to extract the decision boundary from the realization of a
ReLU NN. Both of those methods introduce very natural visualization tool for
ReLU NNs trained on low-dimensional data.