Online estimation of the hand-eye transformation from surgical scenes
Krittin Pachtrachai Francisco Vasconcelos Danail Stoyanov
Hand-eye calibration algorithms are mature and provide accurate
transformation estimations for an effective camera-robot link but rely on a
sufficiently wide range of calibration data to avoid errors and degenerate
configurations. To solve the hand-eye problem in robotic-assisted minimally
invasive surgery and also simplify the calibration procedure by using neural
network method cooporating with the new objective function. We present a neural
network-based solution that estimates the transformation from a sequence of
images and kinematic data which significantly simplifies the calibration
procedure. The network utilises the long short-term memory architecture to
extract temporal information from the data and solve the hand-eye problem. The
objective function is derived from the linear combination of remote centre of
motion constraint, the re-projection error and its derivative to induce a small
change in the hand-eye transformation. The method is validated with the data
from da Vinci Si and the result shows that the estimated hand-eye matrix is
able to re-project the end-effector from the robot coordinate to the camera
coordinate within 10 to 20 pixels of accuracy in both testing dataset. The
calibration performance is also superior to the previous neural network-based
hand-eye method. The proposed algorithm shows that the calibration procedure
can be simplified by using deep learning techniques and the performance is
improved by the assumption of non-static hand-eye transformations.