PEGASUS: Physically Enhanced Gaussian Splatting Simulation System for
  6DOF Object Pose Dataset Generation
Lukas Meyer Floris Erich Yusuke Yoshiyasu Marc Stamminger Noriaki Ando Yukiyasu Domae
We introduce Physically Enhanced Gaussian Splatting Simulation System
(PEGASUS) for 6DOF object pose dataset generation, a versatile dataset
generator based on 3D Gaussian Splatting. Environment and object
representations can be easily obtained using commodity cameras to reconstruct
with Gaussian Splatting. PEGASUS allows the composition of new scenes by
merging the respective underlying Gaussian Splatting point cloud of an
environment with one or multiple objects. Leveraging a physics engine enables
the simulation of natural object placement within a scene through interaction
between meshes extracted for the objects and the environment. Consequently, an
extensive amount of new scenes - static or dynamic - can be created by
combining different environments and objects. By rendering scenes from various
perspectives, diverse data points such as RGB images, depth maps, semantic
masks, and 6DoF object poses can be extracted. Our study demonstrates that
training on data generated by PEGASUS enables pose estimation networks to
successfully transfer from synthetic data to real-world data. Moreover, we
introduce the Ramen dataset, comprising 30 Japanese cup noodle items. This
dataset includes spherical scans that captures images from both object
hemisphere and the Gaussian Splatting reconstruction, making them compatible
with PEGASUS.