Segmentation Framework for Heat Loss Identification in Thermal Images:
  Empowering Scottish Retrofitting and Thermographic Survey Companies
Md Junayed Hasan Eyad Elyan Yijun Yan Jinchang Ren Md Mostafa Kamal Sarker
Retrofitting and thermographic survey (TS) companies in Scotland collaborate
with social housing providers to tackle fuel poverty. They employ ground-level
infrared (IR) camera-based-TSs (GIRTSs) for collecting thermal images to
identi-fy the heat loss sources resulting from poor insulation. However, this
identifica-tion process is labor-intensive and time-consuming, necessitating
extensive data processing. To automate this, an AI-driven approach is
necessary. Therefore, this study proposes a deep learning (DL)-based
segmentation framework using the Mask Region Proposal Convolutional Neural
Network (Mask RCNN) to validate its applicability to these thermal images. The
objective of the framework is to au-tomatically identify, and crop heat loss
sources caused by weak insulation, while also eliminating obstructive objects
present in those images. By doing so, it min-imizes labor-intensive tasks and
provides an automated, consistent, and reliable solution. To validate the
proposed framework, approximately 2500 thermal imag-es were collected in
collaboration with industrial TS partner. Then, 1800 repre-sentative images
were carefully selected with the assistance of experts and anno-tated to
highlight the target objects (TO) to form the final dataset. Subsequently, a
transfer learning strategy was employed to train the dataset, progressively
aug-menting the training data volume and fine-tuning the pre-trained baseline
Mask RCNN. As a result, the final fine-tuned model achieved a mean average
precision (mAP) score of 77.2% for segmenting the TO, demonstrating the
significant po-tential of proposed framework in accurately quantifying energy
loss in Scottish homes.