

Autonomous cars could one day use the system to quickly avoid a potential collision with another car or pedestrian emerging from around a building’s corner or from in between parked cars. Also, in the future, robots that may navigate hospital hallways to make medication or supply deliveries could use the system to avoid hitting people.

In a paper presented at the International Conference on Intelligent Robots and Systems (IROS), the researchers described successful experiments with an autonomous car driving around a parking garage and an autonomous wheelchair navigating hallways.

When sensing and stopping for an approaching vehicle, the car-based system beats traditional LiDAR — which can only detect visible objects — by more than half a second. That may not seem like much, but fractions of a second matter when it comes to fast-moving autonomous vehicles, the researchers say.

For applications where robots are moving around environments with other moving objects or people, the new method can give the robot an early warning that somebody is coming around the corner, so the vehicle can slow down, adapt its path, and prepare in advance to avoid a collision.

So far, the system has only been tested in indoor settings, where robotic speeds are much lower, and lighting conditions are more consistent, making it easier for the system to sense and analyze shadows.

For their work, the researchers built on their system, called “ShadowCam,” which uses computer-vision techniques to detect and classify changes to shadows on the ground. For input, ShadowCam uses sequences of video frames from a camera targeting a specific area, such as the floor in front of a corner. It detects changes in light intensity over time, from image to image, that may indicate something moving away or coming closer. Some of those changes may be difficult to detect or invisible to the naked eye but can be determined by various properties of the object and environment. ShadowCam computes that information and classifies each image as containing a stationary object or a dynamic, moving one. If it gets to a dynamic image, it reacts accordingly.



Visual odometry, used for Mars rovers for example, estimates the motion of a camera in real time by analyzing pose and geometry in sequences of images. The researchers specifically employed “Direct Sparse Odometry” (DSO), which can compute feature points in environments similar to those captured by AprilTags. Essentially, DSO plots features of an environment on a 3D point cloud, and then a computer-vision pipeline selects only the features located in a region of interest, such as the floor near a corner. (Regions of interest were annotated manually beforehand.)

As ShadowCam takes input image sequences of a region of interest, it uses the DSO-image-registration method to overlay all the images from the same viewpoint of the robot. Even as a robot is moving, it’s able to zero in on the exact same patch of pixels where a shadow is located to help it detect any subtle deviations between images.

Next is signal amplification, a technique introduced in the first paper. Pixels that may contain shadows get a boost in color, thereby improving the signal-to-noise ratio. This makes extremely weak signals from shadow changes far more detectable. If the boosted signal reaches a certain threshold — based partly on how much it deviates from other nearby shadows — ShadowCam classifies the image as “dynamic.” Depending on the strength of that signal, the system may tell the robot to slow down or stop. By detecting that signal, you can then be careful because it may be a shadow of some person running from behind the corner or a parked car.



