Enhancing Retrieval Processes for Language Generation with Augmented
  Queries
Julien Pierre Edmond Ghali Kosuke Shima Koichi Moriyama Atsuko Mutoh Nobuhiro Inuzuka
In the rapidly changing world of smart technology, searching for documents
has become more challenging due to the rise of advanced language models. These
models sometimes face difficulties, like providing inaccurate information,
commonly known as "hallucination." This research focuses on addressing this
issue through Retrieval-Augmented Generation (RAG), a technique that guides
models to give accurate responses based on real facts. To overcome scalability
issues, the study explores connecting user queries with sophisticated language
models such as BERT and Orca2, using an innovative query optimization process.
The study unfolds in three scenarios: first, without RAG, second, without
additional assistance, and finally, with extra help. Choosing the compact yet
efficient Orca2 7B model demonstrates a smart use of computing resources. The
empirical results indicate a significant improvement in the initial language
model's performance under RAG, particularly when assisted with prompts
augmenters. Consistency in document retrieval across different encodings
highlights the effectiveness of using language model-generated queries. The
introduction of UMAP for BERT further simplifies document retrieval while
maintaining strong results.