PlankAssembly: Robust 3D Reconstruction from Three Orthographic Views
  with Learnt Shape Programs
Wentao Hu Jia Zheng Zixin Zhang Xiaojun Yuan Jian Yin Zihan Zhou
In this paper, we develop a new method to automatically convert 2D line
drawings from three orthographic views into 3D CAD models. Existing methods for
this problem reconstruct 3D models by back-projecting the 2D observations into
3D space while maintaining explicit correspondence between the input and
output. Such methods are sensitive to errors and noises in the input, thus
often fail in practice where the input drawings created by human designers are
imperfect. To overcome this difficulty, we leverage the attention mechanism in
a Transformer-based sequence generation model to learn flexible mappings
between the input and output. Further, we design shape programs which are
suitable for generating the objects of interest to boost the reconstruction
accuracy and facilitate CAD modeling applications. Experiments on a new
benchmark dataset show that our method significantly outperforms existing ones
when the inputs are noisy or incomplete.