ViFu: Multiple 360$^\circ$ Objects Reconstruction with Clean Background
  via Visible Part Fusion
Tianhan Xu Takuya Ikeda Koichi Nishiwaki
In this paper, we propose a method to segment and recover a static, clean
background and multiple 360$^\circ$ objects from observations of scenes at
different timestamps. Recent works have used neural radiance fields to model 3D
scenes and improved the quality of novel view synthesis, while few studies have
focused on modeling the invisible or occluded parts of the training images.
These under-reconstruction parts constrain both scene editing and rendering
view selection, thereby limiting their utility for synthetic data generation
for downstream tasks. Our basic idea is that, by observing the same set of
objects in various arrangement, so that parts that are invisible in one scene
may become visible in others. By fusing the visible parts from each scene,
occlusion-free rendering of both background and foreground objects can be
achieved.
  We decompose the multi-scene fusion task into two main components: (1)
objects/background segmentation and alignment, where we leverage point
cloud-based methods tailored to our novel problem formulation; (2) radiance
fields fusion, where we introduce visibility field to quantify the visible
information of radiance fields, and propose visibility-aware rendering for the
fusion of series of scenes, ultimately obtaining clean background and
360$^\circ$ object rendering. Comprehensive experiments were conducted on
synthetic and real datasets, and the results demonstrate the effectiveness of
our method.