Control Theoretic Approach to Fine-Tuning and Transfer Learning
Erkan Bayram Shenyu Liu Mohamed-Ali Belabbas Tamer Başar
Given a training set in the form of a paired $(\mathcal{X},\mathcal{Y})$, we
say that the control system $\dot{x} = f(x,u)$ has learned the paired set via
the control $u^*$ if the system steers each point of $\mathcal{X}$ to its
corresponding target in $\mathcal{Y}$. Most existing methods for finding a
control function $u^*$ require learning of a new control function if the
training set is updated. To overcome this limitation, we introduce the concept
of $\textit{tuning without forgetting}$. We develop $\textit{an iterative
algorithm}$ to tune the control function $u^*$ when the training set expands,
whereby points already in the paired set are still matched, and new training
samples are learned. More specifically, at each update of our method, the
control $u^*$ is projected onto the kernel of the end-point mapping generated
by the controlled dynamics at the learned samples. It ensures keeping the end
points for the previously learned samples constant while iteratively learning
additional samples. Our work contributes to the scalability of control methods,
offering a novel approach to adaptively handle training set expansions.