3D Pose Estimation of Tomato Peduncle Nodes using Deep Keypoint
  Detection and Point Cloud
Jianchao Ci Xin Wang David Rapado-Rincón Akshay K. Burusa Gert Kootstra
Greenhouse production of fruits and vegetables in developed countries is
challenged by labor 12 scarcity and high labor costs. Robots offer a good
solution for sustainable and cost-effective 13 production. Acquiring accurate
spatial information about relevant plant parts is vital for 14 successful robot
operation. Robot perception in greenhouses is challenging due to variations in
15 plant appearance, viewpoints, and illumination. This paper proposes a
keypoint-detection-based 16 method using data from an RGB-D camera to estimate
the 3D pose of peduncle nodes, which 17 provides essential information to
harvest the tomato bunches. 18 19 Specifically, this paper proposes a method
that detects four anatomical landmarks in the color 20 image and then
integrates 3D point-cloud information to determine the 3D pose. A 21
comprehensive evaluation was conducted in a commercial greenhouse to gain
insight into the 22 performance of different parts of the method. The results
showed: (1) high accuracy in object 23 detection, achieving an Average
Precision (AP) of AP@0.5=0.96; (2) an average Percentage of 24 Detected Joints
(PDJ) of the keypoints of PhDJ@0.2=94.31%; and (3) 3D pose estimation 25
accuracy with mean absolute errors (MAE) of 11.38o and 9.93o for the relative
upper and lower 26 angles between the peduncle and main stem, respectively.
Furthermore, the capability to handle 27 variations in viewpoint was
investigated, demonstrating the method was robust to view changes. 28 However,
canonical and higher views resulted in slightly higher performance compared to
other 29 views. Although tomato was selected as a use case, the proposed method
is also applicable to 30 other greenhouse crops like pepper.