Experimenting with Large Language Models and vector embeddings in NASA
  SciX
Sergi Blanco-Cuaresma Ioana Ciucă Alberto Accomazzi Michael J. Kurtz Edwin A. Henneken Kelly E. Lockhart Felix Grezes Thomas Allen Golnaz Shapurian Carolyn S. Grant Donna M. Thompson Timothy W. Hostetler Matthew R. Templeton Shinyi Chen Jennifer Koch Taylor Jacovich Daniel Chivvis Fernanda de Macedo Alves Jean-Claude Paquin Jennifer Bartlett Mugdha Polimera Stephanie Jarmak
Open-source Large Language Models enable projects such as NASA SciX (i.e.,
NASA ADS) to think out of the box and try alternative approaches for
information retrieval and data augmentation, while respecting data copyright
and users' privacy. However, when large language models are directly prompted
with questions without any context, they are prone to hallucination. At NASA
SciX we have developed an experiment where we created semantic vectors for our
large collection of abstracts and full-text content, and we designed a prompt
system to ask questions using contextual chunks from our system. Based on a
non-systematic human evaluation, the experiment shows a lower degree of
hallucination and better responses when using Retrieval Augmented Generation.
Further exploration is required to design new features and data augmentation
processes at NASA SciX that leverages this technology while respecting the high
level of trust and quality that the project holds.