Methods for Generating Drift in Text Streams
Cristiano Mesquita Garcia Alessandro Lameiras Koerich Alceu de Souza Britto Jr Jean Paul Barddal
Systems and individuals produce data continuously. On the Internet, people
share their knowledge, sentiments, and opinions, provide reviews about services
and products, and so on. Automatically learning from these textual data can
provide insights to organizations and institutions, thus preventing financial
impacts, for example. To learn from textual data over time, the machine
learning system must account for concept drift. Concept drift is a frequent
phenomenon in real-world datasets and corresponds to changes in data
distribution over time. For instance, a concept drift occurs when sentiments
change or a word's meaning is adjusted over time. Although concept drift is
frequent in real-world applications, benchmark datasets with labeled drifts are
rare in the literature. To bridge this gap, this paper provides four textual
drift generation methods to ease the production of datasets with labeled
drifts. These methods were applied to Yelp and Airbnb datasets and tested using
incremental classifiers respecting the stream mining paradigm to evaluate their
ability to recover from the drifts. Results show that all methods have their
performance degraded right after the drifts, and the incremental SVM is the
fastest to run and recover the previous performance levels regarding accuracy
and Macro F1-Score.