An LLM-Based Digital Twin for Optimizing Human-in-the Loop Systems
Hanqing Yang Marie Siew Carlee Joe-Wong
The increasing prevalence of Cyber-Physical Systems and the Internet of
Things (CPS-IoT) applications and Foundation Models are enabling new
applications that leverage real-time control of the environment. For example,
real-time control of Heating, Ventilation and Air-Conditioning (HVAC) systems
can reduce its usage when not needed for the comfort of human occupants, hence
reducing energy consumption. Collecting real-time feedback on human preferences
in such human-in-the-loop (HITL) systems, however, is difficult in practice. We
propose the use of large language models (LLMs) to deal with the challenges of
dynamic environments and difficult-to-obtain data in CPS optimization. In this
paper, we present a case study that employs LLM agents to mimic the behaviors
and thermal preferences of various population groups (e.g. young families, the
elderly) in a shopping mall. The aggregated thermal preferences are integrated
into an agent-in-the-loop based reinforcement learning algorithm AitL-RL, which
employs the LLM as a dynamic simulation of the physical environment to learn
how to balance between energy savings and occupant comfort. Our results show
that LLMs are capable of simulating complex population movements within large
open spaces. Besides, AitL-RL demonstrates superior performance compared to the
popular existing policy of set point control, suggesting that adaptive and
personalized decision-making is critical for efficient optimization in CPS-IoT
applications. Through this case study, we demonstrate the potential of
integrating advanced Foundation Models like LLMs into CPS-IoT to enhance system
adaptability and efficiency. The project's code can be found on our GitHub
repository.