Linking vision and motion for self-supervised object-centric perception
Kaylene C. Stocking Zak Murez Vijay Badrinarayanan Jamie Shotton Alex Kendall Claire Tomlin Christopher P. Burgess
Object-centric representations enable autonomous driving algorithms to reason
about interactions between many independent agents and scene features.
Traditionally these representations have been obtained via supervised learning,
but this decouples perception from the downstream driving task and could harm
generalization. In this work we adapt a self-supervised object-centric vision
model to perform object decomposition using only RGB video and the pose of the
vehicle as inputs. We demonstrate that our method obtains promising results on
the Waymo Open perception dataset. While object mask quality lags behind
supervised methods or alternatives that use more privileged information, we
find that our model is capable of learning a representation that fuses multiple
camera viewpoints over time and successfully tracks many vehicles and
pedestrians in the dataset. Code for our model is available at
https://github.com/wayveai/SOCS.