Surveillance & Intervention: Collaboration Between a Robot & Fixed Cameras

Cartoon illustrating the components of the system described in the thesis

My thesis describes a collaborative surveillance system comprising a mobile robot (with an on-board panoramic camera) networked to fixed cameras. The mobile robot provides an element of surprise and the potential for interactive investigation into the motives of a trespasser. It provides the possibility of interrogating the subject to ascertain his/her identity using face or fingerprint recognition, failing which it can take steps to detain the subject (by possibly blocking exits) until human help arrives.
The system is capable of autonomously calibrating all cameras to a floorplan of the envi- ronment that is provided to it. The robot is simultaneously localized in the floorplan and in the image plane of the external camera with the best view of the robot. This simultaneity of localization is utilized for the autonomous build up of a homography between the image plane of each camera and the ground plane during an initial start-up phase when the robot explores its environment. The fixed cameras can then use this image plane to ground plane homography to track people on the ground plane.
The robot is able to detect and track people around it while in motion using an on-board
panoramic camera. An operator joystick-controls the robot along a particular route which the robot can later repeat autonomously using visual servoing. A novel anomaly detection algorithm that enables the robot to detect people as anomalies during patrol of this route is presented. Panoramic person detection and tracking outside of trained route- following is also demonstrated in a variety of indoor environments.
When a person is in view of an external camera but is out of visual range of the patrolling robot, the system is able to direct the robot to the person because it knows the ground plane coordinates of the position of the person in the image plane. Once the robot is in proximity of its human target, it is able to pursue the target using on-board sensing.
Experiments have also been conducted on the detection of possible covert behaviour of people with respect to the mobile robot. Their motion is classified as covert (suspicious) or overt (normal) using a metric that determines visibility with respect to the robot’s path during its patrol.


Leave a Reply
