Discovering objects and their attributes is of great importance for autonomous agents to effectively operate in human environments. This task is particularly challenging due to the ubiquitousness of objects and all their nuances in perceptual and semantic detail. In this paper we present an unsupervised approach for learning disentangled representations of objects entirely from unlabeled monocular videos. These continuous representations are not biased by or limited by a discrete set of labels determined by human labelers. The proposed representation is trained with a metric learning loss, where objects with homogeneous features are pushed together, while those with heterogeneous features are pulled apart. We show these unsupervised embeddings allow to discover object attributes and can enable robots to self-supervise in previously unseen environments. We quantitatively evaluate performance on a large-scale synthetic dataset with 12k object models, as well as on a real dataset collected by a robot and show that our unsupervised object understanding generalizes to previously unseen objects. Specifically, we demonstrate the effectiveness of our approach on robotic manipulation tasks, such as pointing at and grasping of objects. An interesting and perhaps surprising finding in this approach is that given a limited set of objects, object correspondences will naturally emerge when using metric learning without requiring explicit positive pairs. The ability to autonomously train to recognize and differentiate previously unseen objects as well as infer general properties and attributes is an important skill for robotic agents. Increased autonomy leads to robustness, one of the main challenges real-world robotics faces. It also renders scaling up data collection practical. Additionally, removing human supervision from the loop has the potential to enable learning richer and less biased continuous representations than ones supervised by a limited set of discrete labels. Unbiased representations can prove useful in unknown future environments different from the ones seen during supervision, a typical challenge for robotics.In this work we present an unsupervised method that learns representations that disentangle perceptual and semantic object attributes such as class, function, and color. We automatically acquire training data by capturing videos with a real robot; a robot base moves around a table to capture objects in various arrangements. Assuming a pre-existing objectness detector, we extract objects from random frames within a same scene containing the same objects, and let the metric learning system decide how to assign positive and negative pairs of embeddings. Representations that generalize across objects naturally emerge despite not being given groundtruth matches. Unlike previous methods, we abstain from employing additional self-supervisory training signals such as tracking or depth. The only inputs to the system are monocular videos. This simplifies data collection and allows our embedding to integrate into existing end-to-end learning pipelines. We demonstrate that a trained Object-Contrastive Network (OCN) embedding allows us to reliably identify object instances based on their visual features such as color and shape. Moreover, we show that objects are also organized along their semantic or functional properties. For example, a cup might not only be associated with other cups, but also with other containers like bowls or vases.The key contributions of this work are: (1) an unsupervised algorithm for learning representations of objects (naturally encoding attributes like class, color, texture and function) which generalize to previously unseen objects; (2) showing monocular videos are sufficient to contrast similar and dissimilar objects pairs naturally without requiring explicit correspondences; (3) demonstrating the autonomy of the system, using a robot from data collection to tasks such as pointing and grasping similar objects to ones presented to it. We introduced a novel unsupervised representation learning algorithm that allows us to differentiate object attributes, such as color, shape, and function. An OCN embedding is learned by contrasting the features of objects captured from two frames of single view camera trajectories of table-top indoor environments. We specifically attend to individual objects by detecting object bounding boxes and leverage a metric learning loss to disentangle subtle variations of object attributes. The resulting embedding space allows to organize objects along multiple dimensions and serves as representation for robotic learning. We show that an OCN embedding can be used on real robotic tasks such as Figure 7 : Robot experiment of grasping the object that is closest to the query object (held by hand). Images on the left are captured by the robot camera, and the images on the right are the video frames from a third person view camera. The leftmost object (black border) is the query object and its nearest neighbors are listed in descending order. The top row and the bottom row show the robot successfully identifies and grasps the object with similar color and shape attribute respectively.grasping and pointing, where it is important to differentiate visual and semantic attributes of individual object instances. Finally, we show that an OCN can be trained efficiently from RGB videos that are automatically obtained from a real robotic agent.
An unsupervised approach for learning disentangled representations of objects entirely from unlabeled monocular videos. Designs a feature representation from video sequences captured from a scene from different view points. Proposal for an unsupervised representation learning method for visual inputs that incorporates a metric learning approach pulling nearest neighbor pairs of image patches close in embedding space while pushing apart other pairs. This paper explores self-supervised learning of object representations, with the main idea to encourage objects with similar features to get further ‘attracted’ to each other.