AGIR: Automating Cyber Threat Intelligence Reporting with Natural
  Language Generation
Filippo Perrina Francesco Marchiori Mauro Conti Nino Vincenzo Verde
Cyber Threat Intelligence (CTI) reporting is pivotal in contemporary risk
management strategies. As the volume of CTI reports continues to surge, the
demand for automated tools to streamline report generation becomes increasingly
apparent. While Natural Language Processing techniques have shown potential in
handling text data, they often struggle to address the complexity of diverse
data sources and their intricate interrelationships. Moreover, established
paradigms like STIX have emerged as de facto standards within the CTI
community, emphasizing the formal categorization of entities and relations to
facilitate consistent data sharing. In this paper, we introduce AGIR (Automatic
Generation of Intelligence Reports), a transformative Natural Language
Generation tool specifically designed to address the pressing challenges in the
realm of CTI reporting. AGIR's primary objective is to empower security
analysts by automating the labor-intensive task of generating comprehensive
intelligence reports from formal representations of entity graphs. AGIR
utilizes a two-stage pipeline by combining the advantages of template-based
approaches and the capabilities of Large Language Models such as ChatGPT. We
evaluate AGIR's report generation capabilities both quantitatively and
qualitatively. The generated reports accurately convey information expressed
through formal language, achieving a high recall value (0.99) without
introducing hallucination. Furthermore, we compare the fluency and utility of
the reports with state-of-the-art approaches, showing how AGIR achieves higher
scores in terms of Syntactic Log-Odds Ratio (SLOR) and through questionnaires.
By using our tool, we estimate that the report writing time is reduced by more
than 40%, therefore streamlining the CTI production of any organization and
contributing to the automation of several CTI tasks.