Classifying spam emails using agglomerative hierarchical clustering and
  a topic-based approach
F. Janez-Martino R. Alaiz-Rodriguez V. Gonzalez-Castro E. Fidalgo E. Alegre
Spam emails are unsolicited, annoying and sometimes harmful messages which
may contain malware, phishing or hoaxes. Unlike most studies that address the
design of efficient anti-spam filters, we approach the spam email problem from
a different and novel perspective. Focusing on the needs of cybersecurity
units, we follow a topic-based approach for addressing the classification of
spam email into multiple categories. We propose SPEMC-15K-E and SPEMC-15K-S,
two novel datasets with approximately 15K emails each in English and Spanish,
respectively, and we label them using agglomerative hierarchical clustering
into 11 classes. We evaluate 16 pipelines, combining four text representation
techniques -Term Frequency-Inverse Document Frequency (TF-IDF), Bag of Words,
Word2Vec and BERT- and four classifiers: Support Vector Machine, N\"aive Bayes,
Random Forest and Logistic Regression. Experimental results show that the
highest performance is achieved with TF-IDF and LR for the English dataset,
with a F1 score of 0.953 and an accuracy of 94.6%, and while for the Spanish
dataset, TF-IDF with NB yields a F1 score of 0.945 and 98.5% accuracy.
Regarding the processing time, TF-IDF with LR leads to the fastest
classification, processing an English and Spanish spam email in and on average,
respectively.