Klemsy - Fotolia


Vulnerability study questions accuracy of CVSS scores

A new study claims social media may be a useful indicator of vulnerability risk and lead to more accurate CVSS scores and prioritization.

A new vulnerability study from cloud security vendor NopSec Inc. questioned the accuracy of the Common Vulnerability Scoring System (CVSS) and claimed social media can offer better indicators for critical vulnerabilities.

NopSec's "2015 State of Vulnerability Risk Management" report looked at more than 65,000 vulnerabilities contained in the National Vulnerability Database over a 20-year period. Among the study's conclusions was that CVSS scores alone are not sufficient metrics to determine risk, and that other factors "need to be considered in order to establish a true technical risk score."

One such factor is the response on social media, specifically Twitter. According to the study, Twitter was among a handful of useful metrics in analyzing vulnerability risks and prioritizing enterprise security responses. NopSec said social media mentions of vulnerabilities are useful in predicting potential risk.

"We found that the response in social media gives an indication of how important the vulnerability is as opposed to the score represented by the CVSS," said Michelangelo Sidagni, CTO of NopSec. "The professionals and the people in your organization, if they feel the vulnerability is important, they talk about it in social media. We wanted to show the correlation between vulnerability criticality and social media mentions, specifically on Twitter."

Without the value of social media response, Sidagni said, a major vulnerability that receives a low to medium CVSS score might not set off the necessary alarms for enterprise security personnel. He cited the Heartbleed vulnerability as an example; the open source encryption flaw revealed last year was seen by experts as one of the worst vulnerabilities they had ever seen, yet Heartbleed only received a CVSS score of 5 out of 10, which is "medium" severity.

But Heartbleed received a lot of attention on social media, Sidagni said, as it should have. NopSec's study found what it classified as a "critical vulnerability" -- such as Heartbleed -- was on average mentioned 748 times on Twitter. Vulnerabilities that received "high" scores from NopSec received 89 mentions on average, while "medium" scores only received 8 mentions.

The Forum of Incident Response and Security Teams (FIRST), which maintains the CVSS, has received criticism and backlash recently over concerns with CVSS scores and, specifically the low CVSS score Heartbleed received.

Some concerns about the system were addressed in version 3 of the CVSS, released May 28, 2015, according to Seth Hanford, chair of the CVSSv3 special interest group at FIRST. "We heard quite a bit of feedback from the vulnerability scoring community that CVSSv2 had a number of issues that needed to be addressed," Hanford said.

Other vendors have tried to use or even push their own vulnerability scoring systems, such as Tripwire in 2014, although experts generally believe a standardized approach is best for vulnerability risk assessments.

"While CVSS can be a powerful indicator, it -- like all generic values -- is generalized," Ben Rothke, senior eGRC consultant at Nettitude Group, said in a blog post for Dell. "For the best efficacy, it needs to be customized to the specific entity using it. But the reality is that most organizations don't do that. They will simply use the information from Rapid7, Qualys, and Tenable without tailoring it to their specific risks and environment."

Rothke argued the CVSS, which is a free and open industry standard, is used as a quick and dirty measure of threat, but that to be truly effective, it needs to be adapted for individual companies.

"What if the company has a vulnerability with a high CVSS score with no exploit for it, but also another vulnerability with a lower CVSS score that does have an exploit," Rothke asked. "Which takes preference?"

Hanford argued each enterprise needs to analyze a vulnerability and CVSS score in the context of their own environment and risk tolerance.

"For example, if there's a vulnerability in Microsoft Windows, CVSS can tell you how damaging it is to that Windows OS -- but a business has to decide how important Windows is to the business," Hanford said. "So if it's Windows on the CEO's desk, that may be more important than Windows in their call center."

NopSec, meanwhile, sees Twitter as the de-facto media for IT and IT security people to share information on. While the study did not identify the specific people tweeting about vulnerabilities, the company found its own vulnerability ratings correlated extremely well with mentions on Twitter.

"It's not that we recommend [using] it as a sole measure of the risk," Sidagni said. "Our feeling is that the CVSS score is not the only measure of the risk. It's actually not an exact measure. The technical risk belongs only to the technical aspect of the vulnerability, but the other aspects [of the vulnerability] include availability of the exploit, the presence of malware, the popularity of the vulnerability in social media, as well as [relevance to] the client base or the customer host."

Next Steps

Find out why the Shellshock vulnerability earned a CVSS score of 10.

Dig Deeper on Emerging cyberattacks and threats

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

Does your organization rely on CVSS scores for vulnerability risk assessments?
We don't use CVSS scores, and are more likely to take seriously things trending on twitter. It sure seems like twitter is more subjective. I expect in a few years we'll have a social media to security list, something like oWASP. It might be a good additional source of ideas.
CVSS as its name stands, is an indicator for severity, not risk, so I would not mix them and analysed them both in a risk assessment. I usually manage technical vulnerabilities as another cycle process inside risk management, usually technical vulnerabilities are been reflected in risk maps as just a couple of risks where We take into account multiple information(publication, exploits, relevance, severity) to estimate probability and impact for that risks.

Another point worth to mention is that CVSSv2 already take into account another criteria such as exploitability and relevance to each organization, in their Temporal and Environmental Metrics, however they are not used widely, because involves knowledge and analysis for each organization.

Social media could be another dimension to take into account, but in my opinion is less reliable than current CVSS metrics.
I tend to hear about new security issues on twitter, yes. That seems faster than these old-school assessments but also more risky. Moving from "I heard about it from my security friends on twitter" to an objective list of trending security issues on twitter (lately) seems like it would take more than a little AI programming ... and it might be an opportunity for someone, something to compete with oWASP.  Not authoritative, but it might be another data point for people.