Balancing Exploration and Exploitation in LLM using Soft RLLF for
  Enhanced Negation Understanding
Ha-Thanh Nguyen Ken Satoh
Finetuning approaches in NLP often focus on exploitation rather than
exploration, which may lead to suboptimal models. Given the vast search space
of natural language, this limited exploration can restrict their performance in
complex, high-stakes domains, where accurate negation understanding and logical
reasoning abilities are crucial. To address this issue, we leverage
Reinforcement Learning from Logical Feedback (RLLF) to create an effective
balance between exploration and exploitation in LLMs. Our approach employs an
appropriate benchmark dataset for training and evaluation, highlighting the
importance of exploration in enhancing negation understanding capabilities. We
compare the performance of our RLLF-enhanced LLMs with baseline models trained
without RLLF, demonstrating the value of this balanced approach. Furthermore,
we showcase the potential of our method in legal AI applications by employing
transfer learning and evaluating its impact on negation understanding. Our
experimental results exhibit the effectiveness of balancing exploration and
exploitation with RLLF in improving LLMs' negation capabilities. This has
implications for the development of more accurate, reliable, and logically
consistent language models in high-stakes domains.