Red Teaming for Large Language Models At Scale: Tackling Hallucinations
  on Mathematics Tasks
Aleksander Buszydlik Karol Dobiczek Michał Teodor Okoń Konrad Skublicki Philip Lippmann Jie Yang
We consider the problem of red teaming LLMs on elementary calculations and
algebraic tasks to evaluate how various prompting techniques affect the quality
of outputs. We present a framework to procedurally generate numerical questions
and puzzles, and compare the results with and without the application of
several red teaming techniques. Our findings suggest that even though
structured reasoning and providing worked-out examples slow down the
deterioration of the quality of answers, the gpt-3.5-turbo and gpt-4 models are
not well suited for elementary calculations and reasoning tasks, also when
being red teamed.