Reverse Engineering Deep ReLU Networks An Optimization-based Algorithm
Mehrab Hamidi
Reverse engineering deep ReLU networks is a critical problem in understanding
the complex behavior and interpretability of neural networks. In this research,
we present a novel method for reconstructing deep ReLU networks by leveraging
convex optimization techniques and a sampling-based approach. Our method begins
by sampling points in the input space and querying the black box model to
obtain the corresponding hyperplanes. We then define a convex optimization
problem with carefully chosen constraints and conditions to guarantee its
convexity. The objective function is designed to minimize the discrepancy
between the reconstructed networks output and the target models output, subject
to the constraints. We employ gradient descent to optimize the objective
function, incorporating L1 or L2 regularization as needed to encourage sparse
or smooth solutions. Our research contributes to the growing body of work on
reverse engineering deep ReLU networks and paves the way for new advancements
in neural network interpretability and security.