Skip to main content

Vision-Based In-Hand Manipulation with Limited Dexterity

Time: Mon 2019-11-25 10.00

Location: Kollegiesalen, Stockholm (English)

Doctoral student: Silvia Cruciani , Robotik, perception och lärande, RPL

Opponent: Joris De Schutter, KU Leuven

Supervisor: Danica Kragic, Numerisk analys och datalogi, NADA


In-hand manipulation is an action that allows for changing the grasp on an object without the need for releasing it. This action is an important component in the manipulation process and helps solving many tasks. Human hands are dexterous instruments suitable for moving an object inside the hand. However, it is not common for robots to be equipped with dexterous hands due to many challenges in control and mechanical design. In fact, robots are frequently equipped with simple parallel grippers, robust but lacking dexterity. This thesis focuses on achieving in-hand manipulation with limited dexterity. The proposed solutions are based only on visual input, without the need for additional sensing capabilities in the robot's hand.

Extrinsic dexterity allows simple grippers to execute in-hand manipulation thanks to the exploitation of external supports. This thesis introduces new methods for solving in-hand manipulation using inertial forces, controlled friction and external pushes as additional supports to enhance the robot's manipulation capabilities. Pivoting is seen as a possible solution for simple grasp changes: two methods, which cope with inexact friction modeling, are reported, and pivoting is successfully integrated in an overall manipulation task. For large scale in-hand manipulation, the Dexterous Manipulation Graph is introduced as a novel representation of the object. This graph is a useful tool for planning how to change a certain grasp via in-hand manipulation. It can also be exploited to combine both in-hand manipulation and regrasping to augment the possibilities of adjusting the grasp. In addition, this method is extended to achieve in-hand manipulation even for objects with unknown shape. To execute the planned object motions within the gripper, dual-arm robots are exploited to enhance the poor dexterity of parallel grippers: the second arm is seen as an additional support that helps in pushing and holding the object to successfully adjust the grasp configuration.

This thesis presents examples of successful executions of tasks where in-hand manipulation is a fundamental step in the manipulation process, showing how the proposed methods are a viable solution for achieving in-hand manipulation with limited dexterity.