Evaluating ML-Based Anomaly Detection Across Datasets of Varied
  Integrity: A Case Study
Adrian Pekar Richard Jozsa
Cybersecurity remains a critical challenge in the digital age, with network
traffic flow anomaly detection being a key pivotal instrument in the fight
against cyber threats. In this study, we address the prevalent issue of data
integrity in network traffic datasets, which are instrumental in developing
machine learning (ML) models for anomaly detection. We introduce two refined
versions of the CICIDS-2017 dataset, NFS-2023-nTE and NFS-2023-TE, processed
using NFStream to ensure methodologically sound flow expiration and labeling.
Our research contrasts the performance of the Random Forest (RF) algorithm
across the original CICIDS-2017, its refined counterparts WTMC-2021 and
CRiSIS-2022, and our NFStream-generated datasets, in both binary and
multi-class classification contexts. We observe that the RF model exhibits
exceptional robustness, achieving consistent high-performance metrics
irrespective of the underlying dataset quality, which prompts a critical
discussion on the actual impact of data integrity on ML efficacy. Our study
underscores the importance of continual refinement and methodological rigor in
dataset generation for network security research. As the landscape of network
threats evolves, so must the tools and techniques used to detect and analyze
them.