Observations on LLMs for Telecom Domain: Capabilities and Limitations
Sumit Soman Ranjani H G
The landscape for building conversational interfaces (chatbots) has witnessed
a paradigm shift with recent developments in generative Artificial Intelligence
(AI) based Large Language Models (LLMs), such as ChatGPT by OpenAI (GPT3.5 and
GPT4), Google's Bard, Large Language Model Meta AI (LLaMA), among others. In
this paper, we analyze capabilities and limitations of incorporating such
models in conversational interfaces for the telecommunication domain,
specifically for enterprise wireless products and services. Using Cradlepoint's
publicly available data for our experiments, we present a comparative analysis
of the responses from such models for multiple use-cases including domain
adaptation for terminology and product taxonomy, context continuity, robustness
to input perturbations and errors. We believe this evaluation would provide
useful insights to data scientists engaged in building customized
conversational interfaces for domain-specific requirements.