MODIPHY: Multimodal Obscured Detection for IoT using PHantom
  Convolution-Enabled Faster YOLO
Shubhabrata Mukherjee Cory Beard Zhu Li
Low-light conditions and occluded scenarios impede object detection in
real-world Internet of Things (IoT) applications like autonomous vehicles and
security systems. While advanced machine learning models strive for accuracy,
their computational demands clash with the limitations of resource-constrained
devices, hampering real-time performance. In our current research, we tackle
this challenge, by introducing "YOLO Phantom", one of the smallest YOLO models
ever conceived. YOLO Phantom utilizes the novel Phantom Convolution block,
achieving comparable accuracy to the latest YOLOv8n model while simultaneously
reducing both parameters and model size by 43%, resulting in a significant 19%
reduction in Giga Floating Point Operations (GFLOPs). YOLO Phantom leverages
transfer learning on our multimodal RGB-infrared dataset to address low-light
and occlusion issues, equipping it with robust vision under adverse conditions.
Its real-world efficacy is demonstrated on an IoT platform with advanced
low-light and RGB cameras, seamlessly connecting to an AWS-based notification
endpoint for efficient real-time object detection. Benchmarks reveal a
substantial boost of 17% and 14% in frames per second (FPS) for thermal and RGB
detection, respectively, compared to the baseline YOLOv8n model. For community
contribution, both the code and the multimodal dataset are available on GitHub.