Implementation of a perception system for autonomous vehicles using a
  detection-segmentation network in SoC FPGA
Maciej Baczmanski Mateusz Wasala Tomasz Kryjak
Perception and control systems for autonomous vehicles are an active area of
scientific and industrial research. These solutions should be characterised by
high efficiency in recognising obstacles and other environmental elements in
different road conditions, real-time capability, and energy efficiency.
Achieving such functionality requires an appropriate algorithm and a suitable
computing platform. In this paper, we have used the MultiTaskV3
detection-segmentation network as the basis for a perception system that can
perform both functionalities within a single architecture. It was appropriately
trained, quantised, and implemented on the AMD Xilinx Kria KV260 Vision AI
embedded platform. By using this device, it was possible to parallelise and
accelerate the computations. Furthermore, the whole system consumes relatively
little power compared to a CPU-based implementation (an average of 5 watts,
compared to the minimum of 55 watts for weaker CPUs, and the small size (119mm
x 140mm x 36mm) of the platform allows it to be used in devices where the
amount of space available is limited. It also achieves an accuracy higher than
97% of the mAP (mean average precision) for object detection and above 90% of
the mIoU (mean intersection over union) for image segmentation. The article
also details the design of the Mecanum wheel vehicle, which was used to test
the proposed solution in a mock-up city.