Hacker News Reader

Comment by freedomben | Hacker News Reader

I thought that was a good question, and certainly one I'd like to know the answer too, but I very much agree it was done rudely. It didn't seem like it was asked in good faith.

Anyone who has maintained large/complex software like this knows that name recognition is worth a ton, and it kind of has to be that way. It's just not practical at all to scrutinize every commit/change as though the committer is an adversary, and particularly when you know the person it is not a reasonable ask. I would bet the truth is basically "yes, we knew him so it didn't get full scrutiny," and honestly that's an honest (but hard to give) answer.

I do hope (perhaps naively) that this (security code reviews) is something AI can get really good at in the future, because that would be a real value add IMHO.

View on HN

This change doesn’t look like it would’ve required much scrutiny, though… it’s three lines long and seems (admittedly as an uninformed outsider) to be obviously wrong. Like, ignoring the fact that Jia Tan happened to be an adversary, I’m kinda shocked that their code review process let it through—unless the standard quite literally is “recognized contributors get rubber-stamped without further review.”

Right, but that was the exact nature of the attack: it's a small commit that doesn't look like it needs a lot of scrutiny. Like, I get that you meant "it wouldn't take much scrutiny to find this" but I mean "it doesn't look like it needs to be scrutinized". Especially because, as mentioned in the first comment of the investigation, the change to an unsafe behavior is deliberately obscured by the formatting of the diff.

It's like Where's Wal(do|ly): once you know where to look, it's obvious, but if you don't even know you're supposed to be looking for it, you may never find it

Right-- A busy maintainer sees a weird looking commit-- but it's three lines long, submitted from a known contributor, and the tests pass. It was very carefully planned to be innocuous-looking enough to not trigger any concerns with a casual once-over (oh, it just changes the way an error is printed) and obfuscated enough to not be obviously malicious because of the diff formatting, and submitted by a reliable known contributor. Each piece was designed to make a rigorous code review as unlikely as they could possibly make it.

Sure, that's not how it's SUPPOSED to happen, but I'll eat my hat if at least 95% of people who've approved a PR at some point couldn't have been walked down that path by a dedicated attacker over time. Hopefully this has been enough of a jolt to make that less likely the next time someone tries it.

People often cite death and taxes as the only certainties in life-- we could easily include human fallibility.

> People often cite death and taxes as the only certainties in life-- we could easily include human fallibility.

What was it ... 80% of aviation accidents due human error?

Sounds about right. Although some number of those could be classified as plane interface errors or process deficiencies, nobody is perfect. Beyond that, in those situations, nobody was deliberately trying to get them to crash the plane!

In the years I worked as a nightclub bouncer, dozens or hundreds of people would try to fool me every night... and sometimes they did! I had a lot of experience foiling them, but they had a lot more time on their hands to scheme whatever thing they were scheming than I had to pay attention to them, individually.

As people pointed out, this was a technically simple attack-- the meat of the attack was psychological and emotional. In practice, particularly smart people are more susceptible to attacks like this because they subconsciously assume they'll catch everything that comes at them, and make a lot of assumptions about the attack vectors of problems based on what they're good at, like the classic XKCD about cryptography vs a wrench.

It’s hard to give that answer because after a security breach has happened, you know you made the wrong choice. It’s a tradeoff, as you said, for practicality.

Sometimes you gamble and lose. The bank doesn’t care that “well there was a good chance I was going to be fine” when it comes time to pay your mortgage.

“But I’m the only one that knows the floor plan!” doesn’t quite cut it. Exit the premises and get some therapy.

> I do hope (perhaps naively) that this (security code reviews) is something AI can get really good at in the future, because that would be a real value add IMHO.

It would offer a good solution, and one that would scale.-

(Until, of course, the AI systems themselves become compromised or weaponized ...

But that is a few arms race cycles away yet.-

Reply on news.ycombinator.com