Humans vs Large Language Models: Judgmental Forecasting in an Era of
  Advanced AI
MAhdi Abolghasemi Odkhishig Ganbold Kristian Rotaru
This study investigates the forecasting accuracy of human experts versus
Large Language Models (LLMs) in the retail sector, particularly during standard
and promotional sales periods. Utilizing a controlled experimental setup with
123 human forecasters and five LLMs, including ChatGPT4, ChatGPT3.5, Bard,
Bing, and Llama2, we evaluated forecasting precision through Mean Absolute
Percentage Error. Our analysis centered on the effect of the following factors
on forecasters performance: the supporting statistical model (baseline and
advanced), whether the product was on promotion, and the nature of external
impact. The findings indicate that LLMs do not consistently outperform humans
in forecasting accuracy and that advanced statistical forecasting models do not
uniformly enhance the performance of either human forecasters or LLMs. Both
human and LLM forecasters exhibited increased forecasting errors, particularly
during promotional periods and under the influence of positive external
impacts. Our findings call for careful consideration when integrating LLMs into
practical forecasting processes.