Synthetic data generation for system identification: leveraging
  knowledge transfer from similar systems
Dario Piga Matteo Rufolo Gabriele Maroni Manas Mejari Marco Forgione
This paper addresses the challenge of overfitting in the learning of
dynamical systems by introducing a novel approach for the generation of
synthetic data, aimed at enhancing model generalization and robustness in
scenarios characterized by data scarcity. Central to the proposed methodology
is the concept of knowledge transfer from systems within the same class.
Specifically, synthetic data is generated through a pre-trained meta-model that
describes a broad class of systems to which the system of interest is assumed
to belong. Training data serves a dual purpose: firstly, as input to the
pre-trained meta model to discern the system's dynamics, enabling the
prediction of its behavior and thereby generating synthetic output sequences
for new input sequences; secondly, in conjunction with synthetic data, to
define the loss function used for model estimation. A validation dataset is
used to tune a scalar hyper-parameter balancing the relative importance of
training and synthetic data in the definition of the loss function. The same
validation set can be also used for other purposes, such as early stopping
during the training, fundamental to avoid overfitting in case of small-size
training datasets. The efficacy of the approach is shown through a numerical
example that highlights the advantages of integrating synthetic data into the
system identification process.