A Performance Evaluation of a Quantized Large Language Model on Various
  Smartphones
Tolga Çöplü Marc Loedi Arto Bendiken Mykhailo Makohin Joshua J. Bouw Stephen Cobb
This paper explores the feasibility and performance of on-device large
language model (LLM) inference on various Apple iPhone models. Amidst the rapid
evolution of generative AI, on-device LLMs offer solutions to privacy,
security, and connectivity challenges inherent in cloud-based models.
Leveraging existing literature on running multi-billion parameter LLMs on
resource-limited devices, our study examines the thermal effects and
interaction speeds of a high-performing LLM across different smartphone
generations. We present real-world performance results, providing insights into
on-device inference capabilities.