Embedded Planogram Compliance Control System
M. Erkin Yücel Serkan Topaloğlu Cem Ünsalan
The retail sector presents several open and challenging problems that could
benefit from advanced pattern recognition and computer vision techniques. One
such critical challenge is planogram compliance control. In this study, we
propose a complete embedded system to tackle this issue. Our system consists of
four key components as image acquisition and transfer via stand-alone embedded
camera module, object detection via computer vision and deep learning methods
working on single board computers, planogram compliance control method again
working on single board computers, and energy harvesting and power management
block to accompany the embedded camera modules. The image acquisition and
transfer block is implemented on the ESP-EYE camera module. The object
detection block is based on YOLOv5 as the deep learning method and local
feature extraction. We implement these methods on Raspberry Pi 4, NVIDIA Jetson
Orin Nano, and NVIDIA Jetson AGX Orin as single board computers. The planogram
compliance control block utilizes sequence alignment through a modified
Needleman-Wunsch algorithm. This block is also working along with the object
detection block on the same single board computers. The energy harvesting and
power management block consists of solar and RF energy harvesting modules with
suitable battery pack for operation. We tested the proposed embedded planogram
compliance control system on two different datasets to provide valuable
insights on its strengths and weaknesses. The results show that our method
achieves F1 scores of 0.997 and 1.0 in object detection and planogram
compliance control blocks, respectively. Furthermore, we calculated that the
complete embedded system can work in stand-alone form up to two years based on
battery. This duration can be further extended with the integration of the
proposed solar and RF energy harvesting options.