Generating 3D Brain Tumor Regions in MRI using Vector-Quantization
  Generative Adversarial Networks
Meng Zhou Matthias W Wagner Uri Tabori Cynthia Hawkins Birgit B Ertl-Wagner Farzad Khalvati
Medical image analysis has significantly benefited from advancements in deep
learning, particularly in the application of Generative Adversarial Networks
(GANs) for generating realistic and diverse images that can augment training
datasets. However, the effectiveness of such approaches is often limited by the
amount of available data in clinical settings. Additionally, the common
GAN-based approach is to generate entire image volumes, rather than solely the
region of interest (ROI). Research on deep learning-based brain tumor
classification using MRI has shown that it is easier to classify the tumor ROIs
compared to the entire image volumes. In this work, we present a novel
framework that uses vector-quantization GAN and a transformer incorporating
masked token modeling to generate high-resolution and diverse 3D brain tumor
ROIs that can be directly used as augmented data for the classification of
brain tumor ROI. We apply our method to two imbalanced datasets where we
augment the minority class: (1) the Multimodal Brain Tumor Segmentation
Challenge (BraTS) 2019 dataset to generate new low-grade glioma (LGG) ROIs to
balance with high-grade glioma (HGG) class; (2) the internal pediatric LGG
(pLGG) dataset tumor ROIs with BRAF V600E Mutation genetic marker to balance
with BRAF Fusion genetic marker class. We show that the proposed method
outperforms various baseline models in both qualitative and quantitative
measurements. The generated data was used to balance the data in the brain
tumor types classification task. Using the augmented data, our approach
surpasses baseline models by 6.4% in AUC on the BraTS 2019 dataset and 4.3% in
AUC on our internal pLGG dataset. The results indicate the generated tumor ROIs
can effectively address the imbalanced data problem. Our proposed method has
the potential to facilitate an accurate diagnosis of rare brain tumors using
MRI scans.