Prompts As Programs: A Structure-Aware Approach to Efficient
  Compile-Time Prompt Optimization
Tobias Schnabel Jennifer Neville
Large language models (LLMs) can now handle longer and more complex inputs,
which facilitate the use of more elaborate prompts. However, prompts often
require some tuning to improve performance for deployment. Recent work has
proposed automatic prompt optimization methods, but as prompt complexity and
LLM strength increase, many prompt optimization techniques are no longer
sufficient and a new approach is needed to optimize {\em meta prompt programs}.
To address this, we introduce SAMMO, a framework for {\em compile-time}
optimizations of metaprompt programs, which represent prompts as structured
objects that allows for a rich set of transformations that can be searched over
during optimization. We show that SAMMO generalizes previous methods and
improves the performance of complex prompts on (1) instruction tuning, (2) RAG
pipeline tuning, and (3) prompt compression, across several different LLMs.
  We make all code available open-source at https://github.com/microsoft/sammo .