Image-based Deep Learning for Smart Digital Twins: a Review
Md Ruman Islam Mahadevan Subramaniam Pei-Chi Huang
Smart Digital twins (SDTs) are being increasingly used to virtually replicate
and predict the behaviors of complex physical systems through continual data
assimilation enabling the optimization of the performance of these systems by
controlling the actions of systems. Recently, deep learning (DL) models have
significantly enhanced the capabilities of SDTs, particularly for tasks such as
predictive maintenance, anomaly detection, and optimization. In many domains,
including medicine, engineering, and education, SDTs use image data
(image-based SDTs) to observe and learn system behaviors and control their
behaviors. This paper focuses on various approaches and associated challenges
in developing image-based SDTs by continually assimilating image data from
physical systems. The paper also discusses the challenges involved in designing
and implementing DL models for SDTs, including data acquisition, processing,
and interpretation. In addition, insights into the future directions and
opportunities for developing new image-based DL approaches to develop robust
SDTs are provided. This includes the potential for using generative models for
data augmentation, developing multi-modal DL models, and exploring the
integration of DL with other technologies, including 5G, edge computing, and
IoT. In this paper, we describe the image-based SDTs, which enable broader
adoption of the digital twin DT paradigms across a broad spectrum of areas and
the development of new methods to improve the abilities of SDTs in replicating,
predicting, and optimizing the behavior of complex systems.