KESDT: knowledge enhanced shallow and deep Transformer for detecting
  adverse drug reactions
Yunzhi Qiu Xiaokun Zhang Weiwei Wang Tongxuan Zhang Bo Xu Hongfei Lin
Adverse drug reaction (ADR) detection is an essential task in the medical
field, as ADRs have a gravely detrimental impact on patients' health and the
healthcare system. Due to a large number of people sharing information on
social media platforms, an increasing number of efforts focus on social media
data to carry out effective ADR detection. Despite having achieved impressive
performance, the existing methods of ADR detection still suffer from three main
challenges. Firstly, researchers have consistently ignored the interaction
between domain keywords and other words in the sentence. Secondly, social media
datasets suffer from the challenges of low annotated data. Thirdly, the issue
of sample imbalance is commonly observed in social media datasets. To solve
these challenges, we propose the Knowledge Enhanced Shallow and Deep
Transformer(KESDT) model for ADR detection. Specifically, to cope with the
first issue, we incorporate the domain keywords into the Transformer model
through a shallow fusion manner, which enables the model to fully exploit the
interactive relationships between domain keywords and other words in the
sentence. To overcome the low annotated data, we integrate the synonym sets
into the Transformer model through a deep fusion manner, which expands the size
of the samples. To mitigate the impact of sample imbalance, we replace the
standard cross entropy loss function with the focal loss function for effective
model training. We conduct extensive experiments on three public datasets
including TwiMed, Twitter, and CADEC. The proposed KESDT outperforms
state-of-the-art baselines on F1 values, with relative improvements of 4.87%,
47.83%, and 5.73% respectively, which demonstrates the effectiveness of our
proposed KESDT.