Classifying patient voice in social media data using neural networks: A
  comparison of AI models on different data sources and therapeutic domains
Giorgos Lysandrou Roma English Owen Vanja Popovic Grant Le Brun Beatrice Alex Elizabeth A. L. Fairley
It is essential that healthcare professionals and members of the healthcare
community can access and easily understand patient experiences in the real
world, so that care standards can be improved and driven towards personalised
drug treatment. Social media platforms and message boards are deemed suitable
sources of patient experience information, as patients have been observed to
discuss and exchange knowledge, look for and provide support online. This paper
tests the hypothesis that not all online patient experience information can be
treated and collected in the same way, as a result of the inherent differences
in the way individuals talk about their journeys, in different therapeutic
domains and or data sources.
  We used linguistic analysis to understand and identify similarities between
datasets, across patient language, between data sources (Reddit, SocialGist)
and therapeutic domains (cardiovascular, oncology, immunology, neurology). We
detected common vocabulary used by patients in the same therapeutic domain
across data sources, except for immunology patients, who use unique vocabulary
between the two data sources, and compared to all other datasets. We combined
linguistically similar datasets to train classifiers (CNN, transformer) to
accurately identify patient experience posts from social media, a task we refer
to as patient voice classification. The cardiovascular and neurology
transformer classifiers perform the best in their respective comparisons for
the Reddit data source, achieving F1-scores of 0.865 and 1.0 respectively. The
overall best performing classifier is the transformer classifier trained on all
data collected for this experiment, achieving F1-scores ranging between 0.863
and 0.995 across all therapeutic domain and data source specific test datasets.