A pipeline for multiple orange detection and tracking with 3-D fruit
  relocalization and neural-net based yield regression in commercial citrus
  orchards
Thiago T. Santos Kleber X. S. de Souza João Camargo Neto Luciano V. Koenigkan Alécio S. Moreira Sônia Ternes
Traditionally, sweet orange crop forecasting has involved manually counting
fruits from numerous trees, which is a labor-intensive process. Automatic
systems for fruit counting, based on proximal imaging, computer vision, and
machine learning, have been considered a promising alternative or complement to
manual counting. These systems require data association components that prevent
multiple counting of the same fruit observed in different images. However,
there is a lack of work evaluating the accuracy of multiple fruit counting,
especially considering (i) occluded and re-entering green fruits on leafy
trees, and (ii) counting ground-truth data measured in the crop field. We
propose a non-invasive alternative that utilizes fruit counting from videos,
implemented as a pipeline. Firstly, we employ CNNs for the detection of visible
fruits. Inter-frame association techniques are then applied to track the fruits
across frames. To handle occluded and re-appeared fruit, we introduce a
relocalization component that employs 3-D estimation of fruit locations.
Finally, a neural network regressor is utilized to estimate the total number of
fruit, integrating image-based fruit counting with other tree data such as crop
variety and tree size. The results demonstrate that the performance of our
approach is closely tied to the quality of the field-collected videos. By
ensuring that at least 30% of the fruit is accurately detected, tracked, and
counted, our yield regressor achieves an impressive coefficient of
determination of 0.85. To the best of our knowledge, this study represents one
of the few endeavors in fruit estimation that incorporates manual fruit
counting as a reference point for evaluation. We also introduce annotated
datasets for multiple orange tracking (MOrangeT) and detection (OranDet),
publicly available to foster the development of novel methods for image-based
fruit counting.