Hacker News Reader

Comment by tptacek | Hacker News Reader

Just a reminder that these severity ratings, while often directionally useful (a 9.x probably is something you should care about if you run the target software; a 2.x is probably not), the ratings themselves are total horseshit and are running industry joke; literally a Ouija board, starting from a hopelessly ambiguous "calculator" that you run and then apply subtle inputs to to get the score where you want it to be.

View on HN

> the ratings themselves are total horseshit and are running industry joke

Any further references to this? I’d love to have something I can send around internally. The people running Tenable tend to treat these numbers as gospel.

Honestly, I'm sure there are better sources - but this might be worthwhile just for the irony: https://www.tenable.com/blog/why-you-need-to-stop-using-cvss...

Well, just pull up a CVSS calculator and start asking yourself questions and playing with it. As a fun game we would have people independently rate bugs on our Slack, mind you these are seasoned information security consultants that have broken a lot of software, and the ratings were hilariously far apart. Ask everyone to rate it low/medium/high/critical and consensus was pretty easy to achieve. Anyway the point is the criteria are quite subjectively. What comes out seems very quantitative, but it’s basically a pile of Jello in terms of how solid it is.

You really should be performing your own risk assessment so you can convert this notional context-free severity rating to your own context-related impact and likelihood to give you your own company risk rating. Ratings over a certain number (to be agree with at board level - in effect your risk appetite) should be treated (risk remediated, removed, transferred to a third party etc etc).

We also do this with pen test reports which suffer the same kinds of issues.

Usually find that sometimes a medium is a critical for us and some times a high is a low for us - based on performing this kind of assessment.

> You really should be performing your own risk assessment so you can convert this notional context-free severity rating to your own context-related impact and likelihood to give you your own company risk rating.

Totally agree, and I do that - check the vulnerability description, see if we’re using that part of the library, etc… it usually works out that we’re not actually affected by whatever the alert is.

This isn't specific to CVSS, it's the same for literally any kind of security-related rating or classification system.

I got out of vulnerability management ~10 years ago and moved into other domains where I didn't really work around CVEs and vulnerability reporting. I dipped my toes back in late last year and couldn't believe that basically nothing had changed.

The attacks are getting more funding and are exponentially more sophisticated. The investment in prevention is meh comparably.

(Just curious) is there something about this that lacks substance? Sure, a CVE rated 8.6 vs a CVE 9.8 may or may not be worse than the other, but is there a better mechanism to communicate severity other than this, or something like "Medium", "High", "Critical", struggling to see your point.

I'm exclusively back on my bullshit about CVSS being a debacle, and hoping that if I whine about it enough, people will stop putting CVSS scores in headlines, because they're meaningless. "Critical" does a fine job.

"Critical" reminds me of the urgency every new feature request receives in the backlog.

Or a Windows Update that requires a reboot.

Sounds like you have a dysfunctional organization.

With apologies to Cardinal Richelieu, give me 6 items from the policies of the most functional large organization, and I will find something in them which will hang it.

If you don't see dysfunction, you're not looking closely enough. That's not to say all organizations are equivalently dysfunctional or that we shouldn't strive for better. I'm just saying that "dysfunctional" isn't a damning accusation.

Absolutely, but that doesn’t mean we shouldn’t be trying to resolve problems all the same. Teams not having control over their work is inefficient.

Ah yes, the ACEI scale: ASAP, Critical, Emergency, and In progress. In Progress being the only thing that actually gets done. The rest is conjecture. ASAP will never get done.

Any earlier than "possible" is obviously "impossible".

But you don't have to do things 'as soon as' they're 'possible'. It does have meaning.

My dad used to joke he kept three piles of papers on his desk at work: "urgent", "very urgent" and "no longer urgent".

I have users forward me critical security alerts all the time - most are not even close to critical.

I like the scores - just ask them - what's the score.

In my mind there are only two factors that matter: the accessibility of the attack vector and the extent of the blast area if the target is compromised.

The funny thing about infosec taxonomies is that actual attacks actively resist classifications like CVSS.

CVSS is generally used on the sysadmin side to figure out what to patch first. Successful attackers generally go after shit that's unpatched. This often means that the most exploited bugs are the ones with the lowest CVSS scores, so new tools and techniques tend to grow around issues that sysadmins consider to not be a big deal.

People eventually catch on that the "no big deal" bugs are getting exploited en-masse, and try to tweak the scoring process, which is why we're on CVSSv3 now with CVSSv4 in the works, and it's just as useless as it's always been.

I hear this argument all the time. It's always followed by some variation of "and that's why we're not going to address this in a timely manner".

These ratings are intended to serve as a baseline for the severity of the issue at hand. If you expect a base CVSS score to provide the answer to "how does this affect me" then you need to learn more about CVSS.

Namely, you should be taking the base CVSS score and including the temporal and environment metrics to actually determine your organizational risk. A base 9.x could easily be driven to low based on the access, exploitability, and CIA requirements for the system at hand.

I always use them as a guide for what we should talk about first to then give them our own internal rating. Typically vuln criticality is very system specific.

Getting a web shell speaks for itself.

Context matters a bit. From what i understand its a plugin that's only sometimes used and would typically (but clearly not always) be not exposed to the internet.

So its a serious vuln, but not as serious as say if it was in a component that was always exposed to the internet in the core part of vmware. I guess vulns can always be worse.

How bad it is depends on how you use vmware. I mean regardless, clearly everyone affected should patch.

Reply on news.ycombinator.com