Self-supervised novel 2D view synthesis of large-scale scenes with
  efficient multi-scale voxel carving
Alexandra Budisteanu Dragos Costea Alina Marcu Marius Leordeanu
The task of generating novel views of real scenes is increasingly important
nowadays when AI models become able to create realistic new worlds. In many
practical applications, it is important for novel view synthesis methods to
stay grounded in the physical world as much as possible, while also being able
to imagine it from previously unseen views. While most current methods are
developed and tested in virtual environments with small scenes and no errors in
pose and depth information, we push the boundaries to the real-world domain of
large scales in the new context of UAVs. Our algorithmic contributions are two
folds. First, we manage to stay anchored in the real 3D world, by introducing
an efficient multi-scale voxel carving method, which is able to accommodate
significant noises in pose, depth, and illumination variations, while being
able to reconstruct the view of the world from drastically different poses at
test time. Second, our final high-resolution output is efficiently self-trained
on data automatically generated by the voxel carving module, which gives it the
flexibility to adapt efficiently to any scene. We demonstrated the
effectiveness of our method on highly complex and large-scale scenes in real
environments while outperforming the current state-of-the-art. Our code is
publicly available: https://github.com/onorabil/MSVC.