Robo360: A 3D Omnispective Multi-Material Robotic Manipulation Dataset
Litian Liang Liuyu Bian Caiwei Xiao Jialin Zhang Linghao Chen Isabella Liu Fanbo Xiang Zhiao Huang Hao Su
Building robots that can automate labor-intensive tasks has long been the
core motivation behind the advancements in computer vision and the robotics
community. Recent interest in leveraging 3D algorithms, particularly neural
fields, has led to advancements in robot perception and physical understanding
in manipulation scenarios. However, the real world's complexity poses
significant challenges. To tackle these challenges, we present Robo360, a
dataset that features robotic manipulation with a dense view coverage, which
enables high-quality 3D neural representation learning, and a diverse set of
objects with various physical and optical properties and facilitates research
in various object manipulation and physical world modeling tasks. We confirm
the effectiveness of our dataset using existing dynamic NeRF and evaluate its
potential in learning multi-view policies. We hope that Robo360 can open new
research directions yet to be explored at the intersection of understanding the
physical world in 3D and robot control.