Zero-shot Bilingual App Reviews Mining with Large Language Models
Jialiang Wei Anne-Lise Courbis Thomas Lambolais Binbin Xu Pierre Louis Bernard Gérard Dray
App reviews from app stores are crucial for improving software requirements.
A large number of valuable reviews are continually being posted, describing
software problems and expected features. Effectively utilizing user reviews
necessitates the extraction of relevant information, as well as their
subsequent summarization. Due to the substantial volume of user reviews, manual
analysis is arduous. Various approaches based on natural language processing
(NLP) have been proposed for automatic user review mining. However, the
majority of them requires a manually crafted dataset to train their models,
which limits their usage in real-world scenarios. In this work, we propose
Mini-BAR, a tool that integrates large language models (LLMs) to perform
zero-shot mining of user reviews in both English and French. Specifically,
Mini-BAR is designed to (i) classify the user reviews, (ii) cluster similar
reviews together, (iii) generate an abstractive summary for each cluster and
(iv) rank the user review clusters. To evaluate the performance of Mini-BAR, we
created a dataset containing 6,000 English and 6,000 French annotated user
reviews and conducted extensive experiments. Preliminary results demonstrate
the effectiveness and efficiency of Mini-BAR in requirement engineering by
analyzing bilingual app reviews. (Replication package containing the code,
dataset, and experiment setups on https://github.com/Jl-wei/mini-bar )