Hacker News Reader

Comment by joeyrideout | Hacker News Reader

I agree. Unfortunately, a lot of security tools get misused in general. (Don't get me started on CVSS!)

I like ASVS and hope that it becomes more popular. Other control standards like CAIQ/CCM are also useful depending on the application.

OWASP SAMM I haven't used yet but I want to have a look! My org uses BSIMM currently. Happy to see an open alternative.

Edit: To be fair, I've noticed "shift left" emerge as a buzzword alongside the popularity of DevOps and DevSecOps. There has been a meaningful improvement in tooling that allows for earlier testing, so I'll concede the new buzzword :)

View on HN

CVSS is worthless; the only way to misuse it is to use it at all.

I’ve never used it, but could you elaborate on why it’s worthless? And what alternatives do you use?

It is an answer to the wrong question.

That is no more helpful in terms of the extra explanation requested than the original response...

Kind of "is it possible to be more vague here?" "yes, yes it is'.

It is a tool to have a discussion, not a solution to risk management. Security is one of the few fields that uses qualitative measurements, and any attempt of applying more quantitative techniques faces a lot of resistance.

CVSS in isolation is pretty good, but for risk management it lacks "context". A higher level framework would be needed for that.

A bug can have a CVSS score of 10 (critical), and still have little to no impact to the business. It's all about context.

I don't think it even ever intended to be more then a data point to consider for risk management but dev

Seconding others; could you elaborate?

I've been involved in some work tangential to the security space, and keep encountering some of those "enterprise security for management framework" buzzwords, things like Lockheed KillChain, VERIS framework and similar. I keep wondering if there's any actual value coming from that direction?

Vulnerability severity is complicated and situational. CVSS is fiddly, but still radically oversimplified. As a result, it's a ouji board metric; it says whatever the rater wants it to say. In standard vulnerability research and assessment practice, XSS is the archetypical sev:medium bug, and SQLI the archetypical sev:hi. Under CVSS, you'll see 8.0+ XSS's and 2.0- SQLIs.

There are sev:crit XSS's! A propagating XSS that can chain to ATO of important accounts would qualify. And there are, once in a blue moon, sev:low SQLIs; for instance, on queries that are already constrained by database permissions. But CVSS doesn't capture these details, and the variance in scores both reflects operator bias (and incompetence) while at the same time failing to describe what those biases are.

It simply doesn't say anything at all.

I do agree CVSS is abused -- especially on NVD -- where they always compute it with "worst case scenario", lacking context and completely messing it up. However, I think CVSS does bring some merit, notably, 1. Forces one to think while assigning values, which means it is more likely that the score will reflect actual severity in most cases. 2. Sets a common tone -- language -- to understanding vulnerability severity, instead of low, medium, high, critical, super critical, which is even worse.

I don't see any scoring system to incorporate the actual operational context, since that is crucial to understanding the real impact of a security issue. CVSS is an attempt, nevertheless, and if you are careful while calculating it, it does add value.

In our organization, we always re-base a vulnerability to an operational context, which makes the score more meaningful.

CVSS has the opposite of merit; it actively confuses people. The same vulnerability found by 3 different teams will have 3 wildly different CVSS scores.

Low, medium, high, critical is actually the interpretive scale provided with CVSS! CVSS isn't getting you away from that; it's just giving you micro-gradations of "medium" or "high". But CVSS isn't even reliable to its first significant figure, let alone into the decimals.

Even worse: The CVSS "calculator" on HackerOne and similar platforms.

I have faithfully filled out a CVSS calculator form for small bugs that I'd consider sev:med and it happily declared them as sev:crit.

It's stupid. CVSS just muddies the water and creates the illusion of a measuring stick where there is none.

I don't bother with CVSS, and none of my clients even notice.

Just treat severity as an enum and explain your decision:

  typedef enum sev { info, low, med, high, crit } sev;

If your justification for severity score can fit in a tweet, and be understood by a layman, mission accomplished.

Doesnt that highlight that the problems is not CVSS, but a better quantitative measurement is needed? What other system is there that would let the 3 people agree on something? Security people seem to be the only once that use qualitative measurements and get away with it... because experts.

Reply on news.ycombinator.com