Similar Data Points Identification with LLM: A Human-in-the-loop
  Strategy Using Summarization and Hidden State Insights
Xianlong Zeng Fanghao Song Ang Liu
This study introduces a simple yet effective method for identifying similar
data points across non-free text domains, such as tabular and image data, using
Large Language Models (LLMs). Our two-step approach involves data point
summarization and hidden state extraction. Initially, data is condensed via
summarization using an LLM, reducing complexity and highlighting essential
information in sentences. Subsequently, the summarization sentences are fed
through another LLM to extract hidden states, serving as compact, feature-rich
representations. This approach leverages the advanced comprehension and
generative capabilities of LLMs, offering a scalable and efficient strategy for
similarity identification across diverse datasets. We demonstrate the
effectiveness of our method in identifying similar data points on multiple
datasets. Additionally, our approach enables non-technical domain experts, such
as fraud investigators or marketing operators, to quickly identify similar data
points tailored to specific scenarios, demonstrating its utility in practical
applications. In general, our results open new avenues for leveraging LLMs in
data analysis across various domains.