Hacker News Reader

Comment by fourcommas | Hacker News Reader

You shouldn't trust someone who misleads with: "Now, 70% of the CVEs originating at Microsoft are memory safety issues, Levick said. “There is no real trend it’s just staying the exact same,” he said. “Despite massive efforts on our part to fix this issue it still seems to be a common thing.”"

There is a trend, and the trend has been a MASSIVE reduction in memory errors. The percentage of security issues that are memory errors says nothing of the totals, which is what matters.

Microsoft could go through 2021 with only one CVE and we'd see articles like this saying "100% of security vulnerabilities are caused by memory errors, we must switch to Rust!"

View on HN

Here is the data from Microsoft Security Response Center on this [1]. Slide 5 shows double as many CVEs as 4-5 years prior, triple as many as 6-8 years prior. Further, slide 10 shows 70% of CVEs are memory safety and that ratio has been constant since 2006. It follows that the absolute number of CVEs which are memory safety is double 4-5 years prior and triple 6-8 years prior, which is inconsistent with a massive reduction.

The magnitudes here are in the many hundreds of CVEs per year from Microsoft (and growing), not "one CVE in 2021". 70% there is not a negligible number.

[1] https://github.com/microsoft/MSRC-Security-Research/blob/mas...

Now divide the number of CVEs in each year by the LoC maintained. You're again looking at completely the wrong number. Line-for-line, C/C++ written today at MSFT/Goog has less memory errors than 10 years ago, and even less exploitation of memory errors. Anyone who lived through the rise and fall of Internet Explorer intuitively knows this.

Dividing by the LoC is a mistaken way of looking at this data. Regardless of the amount of code in a browser or in Windows, an attacker may only need ONE exploitable bug to cause mischief. If the amount of code in Windows grows by a factor X from year to year, the CVEs per LoC better be shrinking by at least factor X (this is where Rust comes in) or else the system is getting less secure. Thus the absolute number is the relevant metric, and indeed is the number reported by Microsoft Security Response Center.

> the CVEs per LoC better be shrinking by at least factor X (this is where Rust comes in) or else the system is getting less secure

Do you really think windows 95 was more secure than win 10? Or that IE6 was more secure than the latest IE? The newer versions are way more secure, it's not even close. Your data is giving you incorrect conclusions, because you're combining and cutting the data in ways that don't make sense.

> There is a trend, and the trend has been a MASSIVE reduction in memory errors. The percentage of security issues that are memory errors says nothing of the totals, which is what matters.

Except that CVE's seem to be increasing over time across the industry, not falling. There is no overall reduction in the numbers.

Indeed, software quality is in decline across the industry.

I think this is more of a "make an idiot-proof system and the world will make a better idiot" type of situation.

How about designing the web around accepting arbitrary code from remote computers and executing it the browser VM.

I agree. While Rust is clearly a step forward in programming languages (and we need a step forward), modern tools built to deal with the issues common in C and C++ like coverage-guided fuzzers, ASan, Valgrind, etc. have made huge improvements in finding memory safety vulnerabilities and other issues. Mitigations have also improved, making it more difficult to exploit memory safety issues.

While I think that unmanaged (i.e. no runtime or interpreter) memory-safe programming languages are a good idea, they're not magic and we shouldn't ignore the other major security tooling improvements recently.

I should note that fuzzers/address sanitizer/Valgrind are very good tools, but they are not perfect. Projects that use them heavily still see issues due to memory-safety bugs. (And yes, memory-safe programming languages are not a magic bullet to fix all bugs; they just help with memory safety…)

> There is a trend, and the trend has been a MASSIVE reduction in memory errors.

Citation?

> the trend has been a MASSIVE reduction in memory errors

What is the evidence for this? I'm not familiar.

Reply on news.ycombinator.com