Transfer learning with generative models for object detection on limited
  datasets
Matteo Paiano Stefano Martina Carlotta Giannelli Filippo Caruso
The availability of data is limited in some fields, especially for object
detection tasks, where it is necessary to have correctly labeled bounding boxes
around each object. A notable example of such data scarcity is found in the
domain of marine biology, where it is useful to develop methods to
automatically detect submarine species for environmental monitoring. To address
this data limitation, the state-of-the-art machine learning strategies employ
two main approaches. The first involves pretraining models on existing datasets
before generalizing to the specific domain of interest. The second strategy is
to create synthetic datasets specifically tailored to the target domain using
methods like copy-paste techniques or ad-hoc simulators. The first strategy
often faces a significant domain shift, while the second demands custom
solutions crafted for the specific task. In response to these challenges, here
we propose a transfer learning framework that is valid for a generic scenario.
In this framework, generated images help to improve the performances of an
object detector in a few-real data regime. This is achieved through a
diffusion-based generative model that was pretrained on large generic datasets,
and is not trained on the task-specific domain. We validate our approach on
object detection tasks, specifically focusing on fishes in an underwater
environment, and on the more common domain of cars in an urban setting. Our
method achieves detection performance comparable to models trained on thousands
of images, using only a few hundreds of input data. Our results pave the way
for new generative AI-based protocols for machine learning applications in
various domains, for instance ranging from geophysics to biology and medicine.