RAYEN: Imposition of Hard Convex Constraints on Neural Networks
Jesus Tordesillas Jonathan P. How Marco Hutter
This paper presents RAYEN, a framework to impose hard convex constraints on
the output or latent variable of a neural network. RAYEN guarantees that, for
any input or any weights of the network, the constraints are satisfied at all
times. Compared to other approaches, RAYEN does not perform a
computationally-expensive orthogonal projection step onto the feasible set,
does not rely on soft constraints (which do not guarantee the satisfaction of
the constraints at test time), does not use conservative approximations of the
feasible set, and does not perform a potentially slow inner gradient descent
correction to enforce the constraints. RAYEN supports any combination of
linear, convex quadratic, second-order cone (SOC), and linear matrix inequality
(LMI) constraints, achieving a very small computational overhead compared to
unconstrained networks. For example, it is able to impose 1K quadratic
constraints on a 1K-dimensional variable with an overhead of less than 8 ms,
and an LMI constraint with 300x300 dense matrices on a 10K-dimensional variable
in less than 12 ms. When used in neural networks that approximate the solution
of constrained optimization problems, RAYEN achieves computation times between
20 and 7468 times faster than state-of-the-art algorithms, while guaranteeing
the satisfaction of the constraints at all times and obtaining a cost very
close to the optimal one.