Research on Named Entity Recognition in Improved transformer with R-Drop
  structure
Weidong Ji Yousheng Zhang Guohui Zhou Xu Wang
To enhance the generalization ability of the model and improve the
effectiveness of the transformer for named entity recognition tasks, the
XLNet-Transformer-R model is proposed in this paper. The XLNet pre-trained
model and the Transformer encoder with relative positional encodings are
combined to enhance the model's ability to process long text and learn
contextual information to improve robustness. To prevent overfitting, the
R-Drop structure is used to improve the generalization capability and enhance
the accuracy of the model in named entity recognition tasks. The model in this
paper performs ablation experiments on the MSRA dataset and comparison
experiments with other models on four datasets with excellent performance,
demonstrating the strategic effectiveness of the XLNet-Transformer-R model.