Exploring the Efficacy of Group-Normalization in Deep Learning Models
  for Alzheimer's Disease Classification
Gousia Habib Ishfaq Ahmed Malik Jameel Ahmad Imtiaz Ahmed Shaima Qureshi
Batch Normalization is an important approach to advancing deep learning since
it allows multiple networks to train simultaneously. A problem arises when
normalizing along the batch dimension because B.N.'s error increases
significantly as batch size shrinks because batch statistics estimates are
inaccurate. As a result, computer vision tasks like detection, segmentation,
and video, which require tiny batches based on memory consumption, aren't
suitable for using Batch Normalization for larger model training and feature
transfer. Here, we explore Group Normalization as an easy alternative to using
Batch Normalization A Group Normalization is a channel normalization method in
which each group is divided into different channels, and the corresponding mean
and variance are calculated for each group. Group Normalization computations
are accurate across a wide range of batch sizes and are independent of batch
size. When trained using a large ImageNet database on ResNet-50, GN achieves a
very low error rate of 10.6% compared to Batch Normalization. when a smaller
batch size of only 2 is used. For usual batch sizes, the performance of G.N. is
comparable to that of Batch Normalization, but at the same time, it outperforms
other normalization techniques. Implementing Group Normalization as a direct
alternative to B.N to combat the serious challenges faced by the Batch
Normalization in deep learning models with comparable or improved
classification accuracy. Additionally, Group Normalization can be naturally
transferred from the pre-training to the fine-tuning phase. .