An FPGA smart camera implementation of segmentation models for drone
  wildfire imagery
Eduardo Guarduño-Martinez Jorge Ciprian-Sanchez Gerardo Valente Vazquez-Garcia Gerardo Rodriguez-Hernandez Adriana Palacios-Rosas Lucile Rossi-Tisson Gilberto Ochoa-Ruiz
Wildfires represent one of the most relevant natural disasters worldwide, due
to their impact on various societal and environmental levels. Thus, a
significant amount of research has been carried out to investigate and apply
computer vision techniques to address this problem. One of the most promising
approaches for wildfire fighting is the use of drones equipped with visible and
infrared cameras for the detection, monitoring, and fire spread assessment in a
remote manner but in close proximity to the affected areas. However,
implementing effective computer vision algorithms on board is often prohibitive
since deploying full-precision deep learning models running on GPU is not a
viable option, due to their high power consumption and the limited payload a
drone can handle. Thus, in this work, we posit that smart cameras, based on
low-power consumption field-programmable gate arrays (FPGAs), in tandem with
binarized neural networks (BNNs), represent a cost-effective alternative for
implementing onboard computing on the edge. Herein we present the
implementation of a segmentation model applied to the Corsican Fire Database.
We optimized an existing U-Net model for such a task and ported the model to an
edge device (a Xilinx Ultra96-v2 FPGA). By pruning and quantizing the original
model, we reduce the number of parameters by 90%. Furthermore, additional
optimizations enabled us to increase the throughput of the original model from
8 frames per second (FPS) to 33.63 FPS without loss in the segmentation
performance: our model obtained 0.912 in Matthews correlation coefficient
(MCC),0.915 in F1 score and 0.870 in Hafiane quality index (HAF), and
comparable qualitative segmentation results when contrasted to the original
full-precision model. The final model was integrated into a low-cost FPGA,
which was used to implement a neural network accelerator.