Entity Recognition from Colloquial Text
Tamara Babaian Jennifer Xu
Extraction of concepts and entities of interest from non-formal texts such as
social media posts and informal communication is an important capability for
decision support systems in many domains, including healthcare, customer
relationship management, and others. Despite the recent advances in training
large language models for a variety of natural language processing tasks, the
developed models and techniques have mainly focused on formal texts and do not
perform as well on colloquial data, which is characterized by a number of
distinct challenges. In our research, we focus on the healthcare domain and
investigate the problem of symptom recognition from colloquial texts by
designing and evaluating several training strategies for BERT-based model
fine-tuning. These strategies are distinguished by the choice of the base
model, the training corpora, and application of term perturbations in the
training data. The best-performing models trained using these strategies
outperform the state-of-the-art specialized symptom recognizer by a large
margin. Through a series of experiments, we have found specific patterns of
model behavior associated with the training strategies we designed. We present
design principles for training strategies for effective entity recognition in
colloquial texts based on our findings.