Seg2Reg: Differentiable 2D Segmentation to 1D Regression Rendering for
  360 Room Layout Reconstruction
Cheng Sun Wei-En Tai Yu-Lin Shih Kuan-Wei Chen Yong-Jing Syu Kent Selwyn The Yu-Chiang Frank Wang Hwann-Tzong Chen
State-of-the-art single-view 360-degree room layout reconstruction methods
formulate the problem as a high-level 1D (per-column) regression task. On the
other hand, traditional low-level 2D layout segmentation is simpler to learn
and can represent occluded regions, but it requires complex post-processing for
the targeting layout polygon and sacrifices accuracy. We present Seg2Reg to
render 1D layout depth regression from the 2D segmentation map in a
differentiable and occlusion-aware way, marrying the merits of both sides.
Specifically, our model predicts floor-plan density for the input
equirectangular 360-degree image. Formulating the 2D layout representation as a
density field enables us to employ `flattened' volume rendering to form 1D
layout depth regression. In addition, we propose a novel 3D warping
augmentation on layout to improve generalization. Finally, we re-implement
recent room layout reconstruction methods into our codebase for benchmarking
and explore modern backbones and training techniques to serve as the strong
baseline. Our model significantly outperforms previous arts. The code will be
made available upon publication.