Sketch2CADScript: 3D Scene Reconstruction from 2D Sketch using Visual
  Transformer and Rhino Grasshopper
Hong-Bin Yang
Existing 3D model reconstruction methods typically produce outputs in the
form of voxels, point clouds, or meshes. However, each of these approaches has
its limitations and may not be suitable for every scenario. For instance, the
resulting model may exhibit a rough surface and distorted structure, making
manual editing and post-processing challenging for humans. In this paper, we
introduce a novel 3D reconstruction method designed to address these issues. We
trained a visual transformer to predict a "scene descriptor" from a single
wire-frame image. This descriptor encompasses crucial information, including
object types and parameters such as position, rotation, and size. With the
predicted parameters, a 3D scene can be reconstructed using 3D modeling
software like Blender or Rhino Grasshopper which provides a programmable
interface, resulting in finely and easily editable 3D models. To evaluate the
proposed model, we created two datasets: one featuring simple scenes and
another with complex scenes. The test results demonstrate the model's ability
to accurately reconstruct simple scenes but reveal its challenges with more
complex ones.