Noise-Robust Fine-Tuning of Pretrained Language Models via External
  Guidance
Song Wang Zhen Tan Ruocheng Guo Jundong Li
Adopting a two-stage paradigm of pretraining followed by fine-tuning,
Pretrained Language Models (PLMs) have achieved substantial advancements in the
field of natural language processing. However, in real-world scenarios, data
labels are often noisy due to the complex annotation process, making it
essential to develop strategies for fine-tuning PLMs with such noisy labels. To
this end, we introduce an innovative approach for fine-tuning PLMs using noisy
labels, which incorporates the guidance of Large Language Models (LLMs) like
ChatGPT. This guidance assists in accurately distinguishing between clean and
noisy samples and provides supplementary information beyond the noisy labels,
thereby boosting the learning process during fine-tuning PLMs. Extensive
experiments on synthetic and real-world noisy datasets further demonstrate the
superior advantages of our framework over the state-of-the-art baselines.