Primal-Dual Continual Learning: Stability and Plasticity through
  Lagrange Multipliers
Juan Elenter Navid NaderiAlizadeh Tara Javidi Alejandro Ribeiro
Continual learning is inherently a constrained learning problem. The goal is
to learn a predictor under a \emph{no-forgetting} requirement. Although several
prior studies formulate it as such, they do not solve the constrained problem
explicitly. In this work, we show that it is both possible and beneficial to
undertake the constrained optimization problem directly. To do this, we
leverage recent results in constrained learning through Lagrangian duality. We
focus on memory-based methods, where a small subset of samples from previous
tasks can be stored in a replay buffer. In this setting, we analyze two
versions of the continual learning problem: a coarse approach with constraints
at the task level and a fine approach with constraints at the sample level. We
show that dual variables indicate the sensitivity of the optimal value with
respect to constraint perturbations. We then leverage this result to partition
the buffer in the coarse approach, allocating more resources to harder tasks,
and to populate the buffer in the fine approach, including only impactful
samples. We derive sub-optimality bounds, and empirically corroborate our
theoretical results in various continual learning benchmarks. We also discuss
the limitations of these methods with respect to the amount of memory available
and the number of constraints involved in the optimization problem.