Herding LLaMaS: Using LLMs as an OS Module
Aditya K Kamath Sujay Yadalam
Computer systems are becoming increasingly heterogeneous with the emergence
of new memory technologies and compute devices. GPUs alongside CPUs have become
commonplace and CXL is poised to be a mainstay of cloud systems. The operating
system is responsible for managing these hardware resources, requiring
modification every time a new device is released. Years of research and
development are sunk into tuning the OS for high performance with each new
heterogeneous device. With the recent explosion in memory technologies and
domain-specific accelerators, it would be beneficial to have an OS that could
provide high performance for new devices without significant effort.
  We propose LLaMaS which can adapt to new devices easily. LLaMaS uses Large
Language Models (LLMs) to extract the useful features of new devices from their
textual description and uses these features to make operating system decisions
at runtime. Adding support to LLaMaS for a new device is as simple as
describing the system and new device properties in plaintext.
  LLaMaS reduces the burden on system administrators to enable easy integration
of new devices into production systems.
  Preliminary evaluation using ChatGPT shows that LLMs are capable of
extracting device features from text and make correct OS decisions based on
those features.