SlotGNN: Unsupervised Discovery of Multi-Object Representations and
  Visual Dynamics
Alireza Rezazadeh Athreyi Badithela Karthik Desingh Changhyun Choi
Learning multi-object dynamics from visual data using unsupervised techniques
is challenging due to the need for robust, object representations that can be
learned through robot interactions. This paper presents a novel framework with
two new architectures: SlotTransport for discovering object representations
from RGB images and SlotGNN for predicting their collective dynamics from RGB
images and robot interactions. Our SlotTransport architecture is based on slot
attention for unsupervised object discovery and uses a feature transport
mechanism to maintain temporal alignment in object-centric representations.
This enables the discovery of slots that consistently reflect the composition
of multi-object scenes. These slots robustly bind to distinct objects, even
under heavy occlusion or absence. Our SlotGNN, a novel unsupervised graph-based
dynamics model, predicts the future state of multi-object scenes. SlotGNN
learns a graph representation of the scene using the discovered slots from
SlotTransport and performs relational and spatial reasoning to predict the
future appearance of each slot conditioned on robot actions. We demonstrate the
effectiveness of SlotTransport in learning object-centric features that
accurately encode both visual and positional information. Further, we highlight
the accuracy of SlotGNN in downstream robotic tasks, including challenging
multi-object rearrangement and long-horizon prediction. Finally, our
unsupervised approach proves effective in the real world. With only minimal
additional data, our framework robustly predicts slots and their corresponding
dynamics in real-world control tasks.