Hacker News Reader

Comment by akersten | Hacker News Reader

Every time I see a fiddly fix like this where we're changing a buffer counter to a postfix instead of a prefix increment, I always think "why are we doing this shit ourselves?"

We should have standardized a C API to safely read/write buffers from files, networks, etc. decades ago. But here we are in 2020 manually moving pointers around - and what a surprise that we're finding "whoops, another off-by-one error, you know how it goes."

No one should be writing new code to read/write bytes. This should be settled. It's like a carpenter building their own nail gun each time they want to build a house.

View on HN

The prefix/postfix change is a red herring, that doesn't do anything (since the expression result isn't used). Which means that the security fix is interspersed with code style changes. Maybe there's a reason to do this here, but it's often not considered good practice because it makes it harder to review the code.

AFAICT, the only real changes are in (new) line 411, resetting the cp variable after the loop, and line 416, ignoring write errors. The former is probably the relevant one.

198 also disables password feedback mode in !tty mode. So, style changes, behavior changes, and security fixes all in one changelist.

This is a small enough diff that I wouldn't complain if given this to review - but I probably also would've reverted the style changes (and would give myself at least 50/50 odds of splitting the changelist in two)

Perhaps I've just been too broken by 1000+ LOC diffs that intersperse multiple behavior changes and style and refactoring and ...

> The prefix/postfix change is a red herring, that doesn't do anything

The prefix/postfix change does something to the reader. It's a distraction trying to figure out if that change was intended, whether the author knows it has no effect, or whether I'm wrong about the change having no effect.

I appreciate the author doing the hard work of fixing this, especially with the code in full view of the public, but if I were an official reviewer I'd ask to get all the unnecessary changes removed.

Better than OpenSSL putting out critical security fixes in a 400kloc diff...

PSA: git add -p and git commit -p exist and are beautiful.

FWIW, sudo uses Mercurial, not git.

https://github.com/sudo-project/sudo is just a mirror.

Doesn't really make a difference, hg offers similar facilities (formerly the record extension, now hg commit -i). There's no index, but it's essentially the same as git commit -p.

> we're changing a buffer counter to a postfix instead of a prefix increment

Maybe I'm missing something, but wasn't that a pointless change? The value of the expression was unused, so post- and pre- are equivalent.

Yes. This thread is blind-leading-the-blind. Threads against C almost always are.

When I was a pentester, out of more than 50 projects I was on, I always found at least a medium-severity bug (a bug which could allow an attacker to pwn the app using nothing but a web browser). Guess how many of those codebases were written in C? Zero.

The smugness is endemic to people outside of security. I was smug too. There are comments I wrote that I cringe at, looking back. Pentesting changes your perspective on things.

C isn't going away. Yes, let's make it safer (by pentesting). No, it's not a mortal sin to write C. I'll find ways to exploit your webapp regardless of the language.

Can I ask what kind of apps you typically worked on?

A big part of why people make a big deal of C is that there are lots of code bases that have been under a lot of scrutiny from security minded code reviewers and pentesters and still keep on yielding more vulnerabilities. Ie it's hard to make C safe even when you really try and are a good programmer.

In contrast IME most pentesters in the biz work on code bases that have seen much less scrutiny and are often in-house and/or "enterprise" apps whose code was written by people who are probably more domain experts than generally good programmers or security-aware developers...

This is a naive and dangerous perspective. You're basically saying that memory safe languages don't solve all problems so it's fine to use unsafe languages.

Don't write code in unsafe languages unless you have to (almost never for new code). Period. You will have some problems either way, but you will have fewer problems in a memory safe language.

Pen testing is not even remotely a good mitigation to depend on against the dangers of a language like C. Its fine to do for defense in depth and to find problems not related to memory safety, but it's no substitute for making a more sensible language choice.

Sudo isn't new code in any sense. "Rewrite the world in memory safe languages" is appealing if you completely discount the time and cost required to do so.

I didn't say that.

And all the new bugs introduced in the process.

That is the classical defense from C camp, going back to the 80's Morris worm.

So you are saying pentesting makes C safe? I wouldn't be so sure, I've seen several 50k+ LOC projects in C that had vulnerabilities even after several audits.

Is it so wrong to recommend a solution that will kill a good chunk of the problems right at the door?

And FYI, you are being pretty smug here as well. The fact that people criticised unrelated parts of the PR in addition to the relevant parts shouldn't give you a free license to dismiss the bigger criticism.

The idea that pentesting will make code safe is pretty crazy... This is basically the same as saying C code doesn't have safety issues as long as you code carefully.

>I always found at least a medium-severity bug >Guess how many of those codebases were written in C? Zero.

So, based on those two comments, you have zero data (anecdotal or other) comparing the rate of bugs in C programs versus other languages since you didn't test any C programs.

Code security is not about eliminating bugs but about lowering the rate they show up at since things always get through and money/time is finite. Pen testing doesn't catch everything and many projects that have been tested still yield bugs. There's also a limit to how much money can be spent on testing so using it find (and document, review, etc.) low hanging bugs due to using C means you're more likely to miss a deeper bug.

Or in other words, spending the same effort to pen test a safe language will leave fewer bugs left over than spending it to pen test an unsafe language. So using an unsafe language is still worse.

I'm reminded of Scott Meyers most important design guideline, which I may be paraphrasing slightly, but it goes:

"An API should be easy to use correctly and hard to use incorrectly"

The dumbest link in the chain is always the guy writing the code, not the programming language, and not the machine executing it. So things should be designed in a way that makes it as hard as possible for programmers to make errors.

e: video is here for those who are interested https://www.youtube.com/watch?v=5tg1ONG18H8

That idea is older. Rusty Russell advocated in 2008 that APIs should be "easy to use and hard to misuse".

https://ozlabs.org/~rusty/index.cgi/tech/2008-03-30.html

https://ozlabs.org/~rusty/index.cgi/tech/2008-04-01.html

Right, this is more broadly applicable than even just APIs or software.

Think about how often you blindly inserted a USB A or B (mini-B, micro-B) plug the wrong way first and had to try again. Why is the wrong way even possible? USB C's plug fixes this.

A huge improvement to railway safety was the invention of Mechanical Interlocking. It was made essentially impossible to set signals in some inherently unsafe ways. When you tried to give a Clear signal to the Express, through a junction a late running Coal Train is already crossing, the huge metal lever you're trying to move to do that won't budge because all your controls are connected via elaborate mechanical interlocking to prevent this mistake. The signal remains at Danger, the Express stops, tragedy is averted.

> Think about how often you blindly inserted a USB A or B (mini-B, micro-B) plug the wrong way first and had to try again. Why is the wrong way even possible? USB C's plug fixes this.

To be honest this isn't really an issue once you notice that the logo side of the USB connector indicates up. I suspect this is mostly an issue because USB connectors have only a tiny self-alignment range (<1 mm), which leads to the "USB exists in a 4D state" meme.

> When I was a pentester, out of more than 50 projects I was on, I always found at least a medium-severity bug

Perhaps you were a good pentester. More likely the code you attacked was rubbish.

I worked on a relatively important Internet-facing web application for about a decade, it was pen tested by a variety of outfits both when it was owned by a small start-up and when it was owned by a huge corporation.

Good design meant that we never saw any pen tester come anywhere close to "pwn the app" in either its public UI or the APIs which were (by my insistence) publicly accessible rather than wasting our time chasing endless "please change the allowed IP ranges" requests through layers of bureaucracy.

My experience was pen testers would bang on the predictable places, try obvious things, maybe trigger some alarms which I then had to pause - and get nowhere. We'd get back a report that said things like if I put in this nonsense email address I get a different error message than this other nonsense email address. OK, that's nice we'll put it on the backlog, did you find any actual security problems? No.

I'd say that using a memory safe language (Java) helped make that happen, IMNSHO as a very good C programmer for whom writing it in Java was not my first choice (nor was C).

I think most of the people you'll find writing code in C today have decades of experience programming. And I think the knowledge and experience they have makes a huge difference in practice when it comes to security engineering.

In comparison, so much modern javascript is written by junior developers - often without a lot of oversight. Junior developers can't "see" security vulnerabilities yet because they haven't learned to look for them. So I'm not surprised there's lots of critical vulnerabilities in modern software. Many coding bootcamps don't bother to teach any security engineering or best practices, and doing consulting work I've (often by chance) caught a disastrous amount of awful code people have tried to push to production simply because they didn't know any better. A lot of it is really simple stuff - guessable database IDs used as session tokens. JSON objects passed from the browser directly into mongodb. Authentication checks accidentally missing from certain admin dashboard APIs. Passwords and credit card information stored in plaintext, handrolled crypto, and so on.

Given the choice between C code and Rust code written by someone who's been programming for 30 years I expect the rust code would be safer. But if I'm asked to choose between C code written by an experienced engineer and javascript code written by a junior engineer, it wouldn't surprise me if it turned out that the C code was on average still more secure.

20+ year programmers are such a tiny tiny cohort compared to recent ones, I think you underestimate this effect.

(Not that 20 years of experience makes you a decent secure C programmer, but that's a different argument)

I think your vastly underestimating the amount of C and C++ written by junior programmers and you're also overestimating the benefits of decades of C experience.

To be honest, as a group, I find C and C++ programmers (even the ones with decades of experience) to be below average in knowledge and skill.

Ouch.

What experiences have given you that opinion?

Occasionally working with people who have spent decades coding almost solely in C or C++ at one of my old jobs. (Not just one person and not just one team.)

I don't remember too many specifics (it was a while ago) other than being brought onboard and having to introduce the tech lead to the "static" keyword.

You would expect developers who put security logic bugs in their code to get C's nuances right while implementing the same application in C?

It is a pointless change in that diff yeah.

Perhaps it's meant to match a style guide that recommends preferring postfix operators? Or perhaps it's editing shrapnel from an earlier version of the changelist, that maybe used the result of that statement?

As far as I can tell, the prefix/postfix thing here is irrelevant and mostly done for style reasons? It’s on its own line and the value of the expression isn’t used, unless I’m missing something obvious.

If anything the solution is more low-level education, not the exact opposite. People who understand these essential basics, e.g. by starting with Asm, are going to be far less likely to make these sorts of mistakes and write less buggy code in general. I mean, a buffer overflow is a very simple concept and it's not hard to understand. How many bytes could be written, and how much storage is reserved? It's really just arithmetic.

The amount of anti-education authoritarian fearmongering in these discussions is disturbing. Then again, given how much everything seems to be rapidly moving in the direction of dystopian corporatocracy, perhaps that's not so surprising.

There's already a few comments about how "safer" languages don't really solve anything. They just push the problems up higher in the stack of abstractions... and when that time comes, what are the chances the "security" fearmongerers will just start blaming something else? It's fundamentally a problem of competence, and doing everything you can to make the world a prison in pursuit of that "perfect security" is really not a good idea. There's no replacement for intelligence.

Security reports from Google and Microsoft's teams prove otherwise.

Also let's not forget that C exploits go back to the 80's, when most of us where doing full application in Assembly.

Supposedly we knew our pointers pretty well.

> Also let's not forget that C exploits go back to the 80's, when most of us where doing full application in Assembly.

I'm not sure this statement adds anything to your argument. "Exploits go back as far as the language was in common use" doesn't really mean anything.

I'm sure Ada has had exploits, especially around its early lifetime. But its a language that's supposed to be safe.

The existence or non-existence of exploits isn't very interesting. The rate that users make vulnerable code is more interesting. The fact that remote-code execution vulnerabilities are still very common in C codebases even after people and the ecosystem have had decades to learn the ins and outs of C is concerning. Maybe the rate of RCE occurrences in C codebases has gone down over time as people have been educated about vulnerabilities more, but I would guess that the rate of RCE vulnerabilities in C codebases is still much higher than the rate ever was for safer languages.

> The existence or non-existence of exploits isn't very interesting. The rate that users make vulnerable code is more interesting.

Yes. Exactly. Exploits having existed for a long time isn't interesting or relevant.

I offered no defence of C. Just that the point wasn't relevant to the argument, one way or the other.

Pjmlp's post was directly responding to this part:

>People who understand these essential basics, e.g. by starting with Asm, are going to be far less likely to make these sorts of mistakes and write less buggy code in general.

Userbinator's post continues and seems to paint the ecosystem as getting worse over time, as if it were much better in this area in the past. Pjmlp's point was that even the original C users who came from assembly, who were presumably more familiar with the essential basics of assembly and memory management, still pumped out vulnerable code as we see today. It doesn't seem like having a userbase made up of experienced assembly programmers is enough for C to be predictably safe.

And C pre-89 is rather different than C11. The language changes, so comparing that which wasn't even specified against what we have to do isn't an argument one way or the other.

Some pre-89 code won't even compile today without putting in considerable effort.

Sure it is, a couple of CVE queries plotting C related exploits per year.

Morris worm was the first widely know C exploit, which still gives us about 30 years of exploits to look into.

Regarding how many exploits other languages have, for starters not memory corruption due to out of bounds access, integer overflow or implicit casts, as Algol derived system languages, since 1961 (8 years before C was even an idea), had checks for those kind of errors.

C only made it to fame, because original UNIX could not be sold and its source code tapes went from university to university.

Had it not been so, probably we would be using Bliss, PL/I, Mesa, PL/S, and not having arguments how good one needs to be to write code that doesn't allow for memory corruption errors.

Exploits having existed for a long time isn't interesting or relevant.

Exploits being consistently discovered over a long time is, and does help your argument.

"Rather different" is a bit much. I've been working with C since the 1989 or so. Other than K&R style function declarations / prototypes, C feels like it has barely changed over the years. It certainly hasn't changed as much as C++.

stdlib was introduced in '89. I specifically said pre-89, because this is what many C programmer's today depend on. malloc and free were brand new in '89 (mmap and friends didn't even exist yet). Instead you'd be playing with calloc and alloca.

Here's one for how safe that was: alloca could not guarantee the pointer it handed you is either valid, or the size you requested.

POSIX.1 only came along in '88 (with our first definition of stdlib). Whilst IO had mostly stabilised by that point, there were still plenty of platforms that didn't use the Bells header (stdio.h).

Files, signals, floating point behaviour, process creation - none of that was standardised yet.

You may be off by a decade. malloc and free existed long before 1988. I can find evidence of it going back as far as 1979 with 7th Edition Unix: https://en.wikipedia.org/wiki/Version_7_Unix http://man.cat-v.org/unix_7th/3/malloc

Most of what was defined by POSIX already existed on common platforms, like SunOS, before it was formally standardized. SunOS 4.0 even had mmap in early 1988! (See https://archive.org/stream/bitsavers_sunsunos4.erenceManual1... )

The fact that pointers and memory management may be so simple on paper, but programmers still routinely mess them up in the same way after decades implies that hoping programmers will finally just start getting it right all the time isn't a good solution. The idea that to solve the problem, everyone else just needs to finally get better, feels like the sort of thing one would say to feel superior to others writing buggy code rather than be a realistic path forward for decreasing the amount of bugs.

I used to have the attitude around vulnerable code of "The problem would be solved if everyone else was just better [like me]". I was desperately trying to prove myself. I'm good at finding and solving tricky bugs, including exploitable memory corruption issues. It felt nice having a thing that I was clearly superior to others in. I loved languages like C where I had a great knowledge of the footguns and I could save the day in. But then I got a good job, I no longer felt undervalued and needing to prove myself at all costs, and my attitude shifted.

I can't write the whole codebase myself or double-check everyone's work. This didn't really matter if I was just concerned with making myself look good, but if the thing I cared about was the product itself and stopping bugs (especially exploitable ones) from getting in to begin with, then it wasn't enough to drop in occasionally to save the day and chide others for not knowing enough about the specific footguns lying around. If there were a bunch of tools that were misused 2/3 of the times they were used in the company, then I could accomplish far more by finding and advocating for safer replacement tools than I could accomplish if I tried to double-check every time the old tool was used and endlessly reminded people that the flesh-magnetic hammer will seek out your thumb unless you know to use some specific swinging strategy.

>There's already a few comments about how "safer" languages don't really solve anything. They just push the problems up higher in the stack of abstractions

A program written in any general purpose language can have high-level bugs like forgetting to check the user's password when they sign in, but only languages with manual memory management make it easy to also have an exploitable memory corruption bug when handling the memory containing the user's password. Solving some kinds of issues is valuable because that can help reduce the count, likelihood, and severity of issues that do happen.

>The amount of anti-education authoritarian fearmongering in these discussions is disturbing. Then again, given how much everything seems to be rapidly moving in the direction of dystopian corporatocracy, perhaps that's not so surprising.

>... and when that time comes, what are the chances the "security" fearmongerers will just start blaming something else?

What possible ulterior motive do you think people advocating safer languages have? I really can't tell if this is parody. Do you think Mozilla made Rust as part of some bigger political narrative, rather than to just make it easier for them to write bug-free code?

It's not about being "better" insomuch as being knowledgeable. My comp. architectures course had a lab where we manually implemented a buffer overflow from seemingly innocuous code, and since learning Rust, I've started writing all my C/C++ code at work to manually check for nullptrs and bounds. If I get it right, branch prediction makes it free. If I don't have it at all, well shit, I might make a critical error in production code!

> It's really just arithmetic.

Understanding these concepts is not the problem. The problem is that even people who understand them very well regularly make mistakes when programming with them, and in C these mistakes have disastrous consequences.

> There's already a few comments about how "safer" languages don't really solve anything. They just push the problems up higher in the stack of abstractions...

The problems that exist higher in the stack of abstractions are present in C programs too. C just creates a lot of additional ways to subvert your program.

Reply on news.ycombinator.com