Design of an basis-projected layer for sparse datasets in deep learning
  training using gc-ms spectra as a case study
Yu Tang Chang Shih Fang Chen
Deep learning (DL) models encompass millions or even billions of parameters
and learn complex patterns from big data. However, not all data are initially
stored in a suitable formation to effectively train a DL model, e.g., gas
chromatography-mass spectrometry (GC-MS) spectra and DNA sequence. These
datasets commonly contain many zero values, and the sparse data formation
causes difficulties in optimizing DL models. A DL module called the
basis-projected layer (BPL) was proposed to mitigate the issue by transforming
the sparse data into a dense representation. The transformed data is expected
to facilitate the gradient calculation and finetuned process in a DL training
process. The dataset, example of a sparse dataset, contained 362 specialty
coffee odorant spectra detected from GC-MS. The BPL layer was placed at the
beginning of the DL model. The tunable parameters in the layer were learnable
projected axes that were the bases of a new representation space. The layer
rotated these bases when its parameters were updated. When the number of the
bases was the same as the original dimension, the increasing percentage of the
F1 scores was 8.56%. Furthermore, when the number was set as 768 (the original
dimension was 490), the increasing percentage of the F1 score was 11.49%. The
layer not only maintained the model performance and even constructed a better
representation space in analyzing sparse datasets.