The VooDoo system allows to track humans in a Human-Robot Interaction context (HRI). 3D and 2D input data from different sensor types like time-of-flight, stereo or monocular cameras is matched against a 3D body model to robustly track human body movements in the vicinity of the robot in real-time. The pose and trajectory of the human interaction partner can then be used for activity and gesture recognition as well as haptic interactions like hand-overs.


VooDoo2 v1.0
Oct 31 2008


Get help and provide feedback in our forums.

About VooDoo

Generally speaking, the VooDoo tracking system is based on

  • the Iterative Closest Point algorithm (ICP)
  • a (human) body model
  • a sensor fusion algorithm
Short introductions to each of these components are given below. Further information can be found on the resources page.

Iterative Closest Point algorithm
The goal of the ICP algorithm is to superimpose two indexed sets of corresponding points which are defined in two different coordinate systems by calculating the translation and rotation that transform the first set into the second. Due to sensor noise, there exists no exact solution to this problem. Instead, one has to minimize the sum of squared distances between corresponding points

where and denote the first and the second set respectively. This assumes that each corresponds to and thus . Since the sensors only yield an unordered list of data points, the correspondence between and must to be computed in the first step. This is done by calculating the closest point on the model for each data point . In the second step, the translation and rotation is estimated and applied to the model. This process is then repeated, until the absolute value of the transformation is below some threshold. Thus, the necessary steps of the ICP algorithm are:
  1. For each data point, calculate the closest point on the body model
  2. Calculate the sum of squared distances between data and model points, which yields
  3. Estimate rotation and translation and apply to the model
  4. Calculate new set of closest points based on the new position of the model
  5. Calculate the sum of squared distances between data and model points, which yields
  6. If the iteration stops, otherwise go to step 3

(Human) body model
The body model used by VooDoo is made up of degenerated cylinders, each of them being defined by 5 parameters (major and minor axis of the top and bottom ellipse and the height of the cylinder) and a proper coordinate system in the center of the bottom ellipse. Starting with the torso as the root node, the overall body model is represented in a hierarchical tree-like structure which stores the degenerated cylinder and the corresponding homogenous transformation matrix of each body part. Note that this body model is not necessarily restricted to humans.

The proposed joint model is based on the idea of connecting two different body parts via elastic bands which allows to model different kinds of joints with varying degrees of freedom:
  • Universal joints with 3 full degrees of freedom (like e.g. the human shoulder) are modeled by a point-to-point correspondence between both body parts (see figure (a)).
  • Hinge joints with 1 full degree of freedom, the others being almost fixed (like e.g. the human hip) are modeled by a set of correspondences which are distributed along a straight line on both body parts(see figure (b)).
  • Elliptic joints have all degrees of freedom highly restricted (like e.g. the human neck or wrist) are modeled by a set of correspondences distributed along an ellipse on both body parts (see figure (c)).

Sensor fusion algorithm
The following illustrates shows the architecture of the VooDoo system, including all input and output channels.

Architecture of the VooDoo system