Hierarchical Mask2Former: Panoptic Segmentation of Crops, Weeds and
  Leaves
Madeleine Darbyshire Elizabeth Sklar Simon Parsons
Advancements in machine vision that enable detailed inferences to be made
from images have the potential to transform many sectors including agriculture.
Precision agriculture, where data analysis enables interventions to be
precisely targeted, has many possible applications. Precision spraying, for
example, can limit the application of herbicide only to weeds, or limit the
application of fertiliser only to undernourished crops, instead of spraying the
entire field. The approach promises to maximise yields, whilst minimising
resource use and harms to the surrounding environment. To this end, we propose
a hierarchical panoptic segmentation method to simultaneously identify
indicators of plant growth and locate weeds within an image. We adapt
Mask2Former, a state-of-the-art architecture for panoptic segmentation, to
predict crop, weed and leaf masks. We achieve a PQ{\dag} of 75.99.
Additionally, we explore approaches to make the architecture more compact and
therefore more suitable for time and compute constrained applications. With our
more compact architecture, inference is up to 60% faster and the reduction in
PQ{\dag} is less than 1%.