Kani: A Lightweight and Highly Hackable Framework for Building Language
  Model Applications
Andrew Zhu Liam Dugan Alyssa Hwang Chris Callison-Burch
Language model applications are becoming increasingly popular and complex,
often including features like tool usage and retrieval augmentation. However,
existing frameworks for such applications are often opinionated, deciding for
developers how their prompts ought to be formatted and imposing limitations on
customizability and reproducibility. To solve this we present Kani: a
lightweight, flexible, and model-agnostic open-source framework for building
language model applications. Kani helps developers implement a variety of
complex features by supporting the core building blocks of chat interaction:
model interfacing, chat management, and robust function calling. All Kani core
functions are easily overridable and well documented to empower developers to
customize functionality for their own needs. Kani thus serves as a useful tool
for researchers, hobbyists, and industry professionals alike to accelerate
their development while retaining interoperability and fine-grained control.