The security scanner that cried wolf

The question is "do you want to know about a valid CVE if there's no patch available"?

For high security environments, the answer might be yes (you want to evaluate individual patches to see if they impact your application)

For many environments the answer is likely (as is the case with OP) no, you only want to know about issues for which there is an updated package available.

Interestingly (well to me :) ) more established scanning tools (e.g. Tenable Nessus) default to "no" on this question and container scanning tools mostly default to "yes". (I did an example comparison https://raesene.github.io/blog/2020/11/22/When_Is_A_Vulnerab...)

The important part is that your organization makes an informed choice about which works best for them.

As a sidenote there are also differences in severities and how vulnerabilities are counted (e.g. if you have multiple issues in a package do you count each one or just count once for that package) which can lead to different numbers of issues being reported by different scanners. https://raesene.github.io/blog/2020/06/21/Container_Vulnerab...

I've seen multiple known OSS authors speaking publicly unfavorably about these security scanners. It seems everyone is offering a "solution" to the vulnerability issue with Open Source, but they do it in a way that at best it's a reminder to update your dependencies (as noted in the article) and at worst it's a burden to OSS authors.

Their interests are also very misaligned with everyone else in the field; best security practices are to communicate security issues with the author privately, have you or them make a fix with a reasonable timeline and then publish a fix for everyone (or reveal the issues publicly after X days). That'd help both authors and users, but unfortunately it doesn't help selling the security tools.

Instead, due to this misalignment of incentives, the current player's workflow include bad practices like:

- First time someone notices a potential vulnerability they will register it in their system, and then if you are lucky you'll get a public issue. Responsive disclosure sounds like "no publicity for us so nah".

- Paint features as security vulnerabilities. Example: your "arbitrary command execution tool" can "execute arbitrary commands" or "your JSON reading tool can read JSON". Well, thanks, that's totally useless.

- Tell users that your dev libraries, that never end up in production, have critical vulnerabilities. Sure if my CSS compilation tool has one it won't matter since it doesn't end up in the production server (unless it's a vuln. in the output).

Edit: for a positive note, these are the current champions IMHO that are helping the ecosystem in this field (besides authors themselves ofc):

- Github's free Actions for OSS. This has been amazingly helpful for testing for free and easily, and basically my turning point on believing that Microsoft has evolved for the better.

- Hundreds or thousands of volunteer fellow OSS authors disclosing vulnerabilities privately. They are mostly anonymous, but def a huge force.

- Companies releasing open source, since they are in a better position to offer big well-rounded libraries that reduce your dependency graph.

If an automated process really could find vulnerabilities to the same fidelity as a human pentester, that would be groundbreaking. In most cases, companies don't want to fork out the cost for real security, so they run these useless scanners instead.

This reminds me of the era when everyone thought that the way to prevent users from doing things that permanently delete data and can't be undone was to throw up a big red "Are you sure?" confirmation. After a while users use got habituated to clicking "Yes", which made the warnings useless.

One example: I enabled AWS Security Hub to have some sort of security score on my account and although it found a few interesting, it generates way too much noise to be usable.

Microsoft also has this sort of security score for Azure and Office 365 environments and they have the same issues.

One of the craziest recent examples was a scan using a tool called Twistlock. Many of our images are built from an upstream image that may have outdated apt dependencies, so one of the first things we do is upgrade them. Twistlock flagged _every instance_ of this because "Package binaries should not be altered" (in other words, between subsequent layers in an image). I am baffled how anyone at Twistlock decided that this was a useful thing for their product to detect, or why any Twistlock customer trusts it given issues like this.

Security is a dynamic process, and scanners are a tool to help compare the current state of your systems with the policies you have set. Out of the box, the scanners can not be aware of your policies and will by necessity produce false positives.

But while "no patch available" is a problem, I think it's the wrong problem to think about. One obvious concern about that line of thinking is that an exposure is an exposure regardless of whether there's a patch available.

The bigger problem, not just with container scanners but with all sorts of scanners (Node dependency scanners are another good example) is that there's a huge incentive to "enumerate" vulnerabilities with CVEs, and most of these vulnerabilities are meaningless.

I spent a couple years at my previous gig triaging security scanner results. Like pretty much everyone who does this professionally, I got so I could do this pretty quickly, and without much though about my actual environment; whole classes of vulnerabilities are pretty much garbage, and most people would be better off if their scanner required a special flag to alert on them.

What you really need to know is, given a vulnerability, is it likely to constitute an exposure in my environment? Clearly, a scanner vendor can't do this analysis perfectly. But they can do more than the zero they do now.

I started my computer security career at a firm called Secure Networks. We sold a security scanner called Ballista. We competed with a much-better-known firm called ISS, who sold the eponymous ISS scanner. The big debate between us and in the market was how to "count" vulnerabilities; ISS, for instance, had a huge collection of Windows registry best-practices rules, and claimed every one of them as (in effect) a vulnerability, while we tried to claim only what our scanner could actually exploit.

We lost that debate, obviously, and the CVE system followed shortly thereafter.

I think the general thing to keep in mind is that, for the most part, modern security scanner teams are pretty small. They're driven off vendorsec and CVE feeds because that's automatable. Like ISS vs SNI, they compete with each other, and their figure of merit is often "how many", not "how important" --- a metric they're not staffed to generate anyways. I wouldn't put much stock in their results.

Is IBM/redhat paying someone here? I don't understand why it would be different.

Redhat came up with 0 security issues, whilst Debian came up with 63 illegal/wontfix issues. Just do your homework, Debian!