Puzzle Solving using Reasoning of Large Language Models: A Survey
Panagiotis Giadikiaroglou Maria Lymperaiou Giorgos Filandrianos Giorgos Stamou
Exploring the capabilities of Large Language Models (LLMs) in puzzle solving
unveils critical insights into their potential and challenges in artificial
intelligence, marking a significant step towards understanding their
applicability in complex reasoning tasks. This survey leverages a unique
taxonomy -- dividing puzzles into rule-based and rule-less categories -- to
critically assess LLMs through various methodologies, including prompting
techniques, neuro-symbolic approaches, and fine-tuning. Through a critical
review of relevant datasets and benchmarks, we assess LLMs' performance,
identifying significant challenges in complex puzzle scenarios. Our findings
highlight the disparity between LLM capabilities and human-like reasoning,
particularly in those requiring advanced logical inference. The survey
underscores the necessity for novel strategies and richer datasets to advance
LLMs' puzzle-solving proficiency and contribute to AI's logical reasoning and
creative problem-solving advancements.