A 3M-Hybrid Model for the Restoration of Unique Giant Murals: A Case
  Study on the Murals of Yongle Palace
Jing Yang Nur Intan Raihana Ruhaiyem Chichun Zhou
The Yongle Palace murals, as valuable cultural heritage, have suffered
varying degrees of damage, making their restoration of significant importance.
However, the giant size and unique data of Yongle Palace murals present
challenges for existing deep-learning based restoration methods: 1) The
distinctive style introduces domain bias in traditional transfer learning-based
restoration methods, while the scarcity of mural data further limits the
applicability of these methods. 2) Additionally, the giant size of these murals
results in a wider range of defect types and sizes, necessitating models with
greater adaptability. Consequently, there is a lack of focus on deep
learning-based restoration methods for the unique giant murals of Yongle
Palace. Here, a 3M-Hybrid model is proposed to address these challenges.
Firstly, based on the characteristic that the mural data frequency is prominent
in the distribution of low and high frequency features, high and low frequency
features are separately abstracted for complementary learning. Furthermore, we
integrate a pre-trained Vision Transformer model (VIT) into the CNN module,
allowing us to leverage the benefits of a large model while mitigating domain
bias. Secondly, we mitigate seam and structural distortion issues resulting
from the restoration of large defects by employing a multi-scale and
multi-perspective strategy, including data segmentation and fusion.
Experimental results demonstrate the efficacy of our proposed model. In
regular-sized mural restoration, it improves SSIM and PSNR by 14.61% and 4.73%,
respectively, compared to the best model among four representative CNN models.
Additionally, it achieves favorable results in the final restoration of giant
murals.