SG-GAN: Fine Stereoscopic-Aware Generation for 3D Brain Point Cloud
  Up-sampling from a Single Image
Bowen Hu Baiying Lei Shuqiang Wang
In minimally-invasive brain surgeries with indirect and narrow operating
environments, 3D brain reconstruction is crucial. However, as requirements of
accuracy for some new minimally-invasive surgeries (such as brain-computer
interface surgery) are higher and higher, the outputs of conventional 3D
reconstruction, such as point cloud (PC), are facing the challenges that sample
points are too sparse and the precision is insufficient. On the other hand,
there is a scarcity of high-density point cloud datasets, which makes it
challenging to train models for direct reconstruction of high-density brain
point clouds. In this work, a novel model named stereoscopic-aware graph
generative adversarial network (SG-GAN) with two stages is proposed to generate
fine high-density PC conditioned on a single image. The Stage-I GAN sketches
the primitive shape and basic structure of the organ based on the given image,
yielding Stage-I point clouds. The Stage-II GAN takes the results from Stage-I
and generates high-density point clouds with detailed features. The Stage-II
GAN is capable of correcting defects and restoring the detailed features of the
region of interest (ROI) through the up-sampling process. Furthermore, a
parameter-free-attention-based free-transforming module is developed to learn
the efficient features of input, while upholding a promising performance.
Comparing with the existing methods, the SG-GAN model shows superior
performance in terms of visual quality, objective measurements, and performance
in classification, as demonstrated by comprehensive results measured by several
evaluation metrics including PC-to-PC error and Chamfer distance.