A Novel Approach To User Agent String Parsing For Vulnerability Analysis
  Using Mutli-Headed Attention
Dhruv Nandakumar Sathvik Murli Ankur Khosla Kevin Choi Abdul Rahman Drew Walsh Scott Riede Eric Dull Edward Bowen
The increasing reliance on the internet has led to the proliferation of a
diverse set of web-browsers and operating systems (OSs) capable of browsing the
web. User agent strings (UASs) are a component of web browsing that are
transmitted with every Hypertext Transfer Protocol (HTTP) request. They contain
information about the client device and software, which is used by web servers
for various purposes such as content negotiation and security. However, due to
the proliferation of various browsers and devices, parsing UASs is a
non-trivial task due to a lack of standardization of UAS formats. Current
rules-based approaches are often brittle and can fail when encountering such
non-standard formats. In this work, a novel methodology for parsing UASs using
Multi-Headed Attention Based transformers is proposed. The proposed methodology
exhibits strong performance in parsing a variety of UASs with differing
formats. Furthermore, a framework to utilize parsed UASs to estimate the
vulnerability scores for large sections of publicly visible IT networks or
regions is also discussed. The methodology present here can also be easily
extended or deployed for real-time parsing of logs in enterprise settings.