Jia Tan "JiaT75": Added error text to warning when untaring with bsdtar (2021) | Hacker News Reader

Jia Tan "JiaT75": Added error text to warning when untaring with bsdtar (2021)

github.com

160 pointsBluestein98 comments

GitHub "community" is just awful. There are people trying to get real work done in that thread, but then there are all these random bystanders piling up to throw in their comments which range from useless to actively harmful and distracting.

And it's not an isolated case, this happens pretty much always when some issues attracts attention on GH.

Can't we respect the project and give the people there space to work, and leave the peanut gallery commenting to reddit/hn/twitter/whatev.

This is why a lot of projects use other tools for bug tracking and merging, e.g. bugzilla, youtrack, atlassian etc.

The opposite. Whenever some project migrates to an inferior alternative like Github or Discord they always claim a number of arguments which boil down to "it's where people go these days" (e.g. less friction for newcomers, most people have an account there already, larger community, whatever excuse you can come up).

So I say they are getting exactly what they wanted to get.

> less friction for newcomers

This is exactly the problem. If you want your resort be vandalized, build a nice road to it.

And some orgs are even eager to fuel this with things like Hacktoberfest.

Apache Foundation maintains a lengthy list of capable projects ... shout out to TRAC first

I like the comment that started with "way too many arm chair 'researchers' in this thread" and then goes on to rudely say that the maintainers are doing a bad job because they merged in the original changes by Jia Tan.

What are you sitting on, if not an arm chair? We all agree that the xz attack was of unparalleled sophistication and complexity, spread carefully over years, funded by a State. Many people were taken in so how is it helpful to pile on Jia Tan's primary victims?

The question is still very relevant. Why are PR like this merged to begin with?

The malicious commit was designed to be confusing, as noted in the first comment of the investigation:

> but calls to safe_fprintf were replaced with calls to the unsafe fprintf. The diff doesn't make this obvious due to the removal of a newline in a parameter list.

It wasn't noticed because it was specifically designed not to be obvious.

because there are hundreds of thousands of programmers who don't have the need for "better" and are willing to put up with C and shell scripts, and "small incremental changes"

I thought that was a good question, and certainly one I'd like to know the answer too, but I very much agree it was done rudely. It didn't seem like it was asked in good faith.

Anyone who has maintained large/complex software like this knows that name recognition is worth a ton, and it kind of has to be that way. It's just not practical at all to scrutinize every commit/change as though the committer is an adversary, and particularly when you know the person it is not a reasonable ask. I would bet the truth is basically "yes, we knew him so it didn't get full scrutiny," and honestly that's an honest (but hard to give) answer.

I do hope (perhaps naively) that this (security code reviews) is something AI can get really good at in the future, because that would be a real value add IMHO.

This change doesn’t look like it would’ve required much scrutiny, though… it’s three lines long and seems (admittedly as an uninformed outsider) to be obviously wrong. Like, ignoring the fact that Jia Tan happened to be an adversary, I’m kinda shocked that their code review process let it through—unless the standard quite literally is “recognized contributors get rubber-stamped without further review.”

Right, but that was the exact nature of the attack: it's a small commit that doesn't look like it needs a lot of scrutiny. Like, I get that you meant "it wouldn't take much scrutiny to find this" but I mean "it doesn't look like it needs to be scrutinized". Especially because, as mentioned in the first comment of the investigation, the change to an unsafe behavior is deliberately obscured by the formatting of the diff.

It's like Where's Wal(do|ly): once you know where to look, it's obvious, but if you don't even know you're supposed to be looking for it, you may never find it

Right-- A busy maintainer sees a weird looking commit-- but it's three lines long, submitted from a known contributor, and the tests pass. It was very carefully planned to be innocuous-looking enough to not trigger any concerns with a casual once-over (oh, it just changes the way an error is printed) and obfuscated enough to not be obviously malicious because of the diff formatting, and submitted by a reliable known contributor. Each piece was designed to make a rigorous code review as unlikely as they could possibly make it.

Sure, that's not how it's SUPPOSED to happen, but I'll eat my hat if at least 95% of people who've approved a PR at some point couldn't have been walked down that path by a dedicated attacker over time. Hopefully this has been enough of a jolt to make that less likely the next time someone tries it.

People often cite death and taxes as the only certainties in life-- we could easily include human fallibility.

> People often cite death and taxes as the only certainties in life-- we could easily include human fallibility.

What was it ... 80% of aviation accidents due human error?

Sounds about right. Although some number of those could be classified as plane interface errors or process deficiencies, nobody is perfect. Beyond that, in those situations, nobody was deliberately trying to get them to crash the plane!

In the years I worked as a nightclub bouncer, dozens or hundreds of people would try to fool me every night... and sometimes they did! I had a lot of experience foiling them, but they had a lot more time on their hands to scheme whatever thing they were scheming than I had to pay attention to them, individually.

As people pointed out, this was a technically simple attack-- the meat of the attack was psychological and emotional. In practice, particularly smart people are more susceptible to attacks like this because they subconsciously assume they'll catch everything that comes at them, and make a lot of assumptions about the attack vectors of problems based on what they're good at, like the classic XKCD about cryptography vs a wrench.

> I do hope (perhaps naively) that this (security code reviews) is something AI can get really good at in the future, because that would be a real value add IMHO.

It would offer a good solution, and one that would scale.-

(Until, of course, the AI systems themselves become compromised or weaponized ...

But that is a few arms race cycles away yet.-

It’s hard to give that answer because after a security breach has happened, you know you made the wrong choice. It’s a tradeoff, as you said, for practicality.

Sometimes you gamble and lose. The bank doesn’t care that “well there was a good chance I was going to be fine” when it comes time to pay your mortgage.

“But I’m the only one that knows the floor plan!” doesn’t quite cut it. Exit the premises and get some therapy.

I recently removed the arm rests from my shitty IKEA chair (to apply WD40 silicone, because the standard cheap mechanism was creaking every time I leaned back, even if it was locked), and decided not to put them back, because they incentivized slouching

so akcheually some people are sitting on a block of concrete!

theorizing that it was wrong to merge in itself is not victim blaming. but of course piling up in GH discussions and issues unconstructively, and just expressing opinions is bullying.

It seems that blaming victims can really provide some power thrill, to which human can easily become addicted to.

Raising competent empathic well balanced individuals is difficult, to say the least. And it’s not like the so called world leader elites really show they are some paragons of these traits.

Absolute worst is when you see the "Hi from HN :wave:" comments when Github issues/PRs linked to from here.

Maintainers can 'lock down' the issue to just projects members, but that's very much an after-the-fact thing.

People treat Github like social media

It is a social medium by definition, tbf.

This.-

  PS. And, for some people it might undoubtedly be *all* their "social" ...

That's because it kinda is a social network. You can "follow" people after all.

It seems like GitHub encourages that? I mean, emoji reacts to issue comments? Not really sure what the point of that is besides "driving engagement".

I guess the argument for it is that it lets people easily "get involved"? Seems like there's some merit to making it easy for users to leave feedback. Maybe thumbs-up or thumbs-down on an issue really is valuable feedback in some situations. I'm torn between saying the social features are bad because they lower the barrier for low quality engagement, and saying that even low quality engagement can lead to valuable insights.

Github is social media

Github launched with the tag line "social code hosting" in 2008:

https://web.archive.org/web/20081216011059/https://github.co...

Which changed to "social coding" later:

https://web.archive.org/web/20110217073759/https://github.co...

"social coding" eventually moved to the page title, and disappeared from the page itself:

https://web.archive.org/web/20120202143623/https://github.co...

"Social code hosting" and "social coding" are not on the front page anymore, but they definitely kept all the social features.

Because it's September, 1993.

GH needs a system like most forum communities have where comments can be put in a moderation / approval queue before being public.

Not a bad idea. I guess the UX is not hard, but likely it would clash with GH's bias towards openness/transparency/public record. Possibly risk making moderation 1st class citizen, they may fear this would detract from the main purpose of collaborative code production. Yet...it could help that. But I understand it's a tricky nuance.

So, they have comment minimization by assigned moderators. Or you can just delete / edit comments and issues. Obviously not as powerful as a pre-screening queue. Less work tho!

They'd end up like SO ...

Liberal use of the GH moderations tools, like block user, and interaction limits, is your friend.

I'm confused why this is being posted now? This thread appears to have taken place in the days immediately following the original XZ discovery, with no new activity since very early April. It was discussed heavily at the time that Jia Tan had made contributions to other projects and that those were being investigated as well.

Is there something new here I missed, or some additional context that makes this specific commit relevant right now?

Since the posted thread has been locked since April, even if there’s new information it can’t be on the posted page. I suspect a lot of votes come from people thinking there’s significant new information (otherwise why would this suddenly be #1?) when there’s none.

Karma farming.

Ok, but to what end? Is there some karma-to-dollars pipeline that I don't know about? There a bunch of other platforms that superficially seem like much softer targets with more obvious payoffs.

Like, if we put it in the classic context of

1. Farm Karma

2. ?

3. Profit!

I'm not clear on step 2. What's step 2

And of course that pre-supposes malice (or at least greed), which is in violation of Hanlon's Razor.

Coincidentally, Reddit (YC05) IPO'ed in March so I'm sure you'll find plenty of analysis on that.

Gallowboob reportedly got paid <https://knowyourmeme.com/memes/people/gallowboob>.

It used to be called Curation, Marketing, or Expert Advice but it's been algorithmified to death.

Sorry, I don't understand the point you're making here. Are you saying that karma farming on HN leads to successful IPOs? Or are you saying that karma farming in general can be profitable? Because both of those are what I was trying to speak to when I said that I feel like there are much softer targets than HN: it seems much easier to me to profit from karma farming on other platforms than it would be here. Maybe I'm just not engaged enough and/or naive, but I don't think of even high-karma users on HN as being Influencers. Like, I don't see myself spending money on something specifically because tptacek endorsed it.

On Instagram, it makes sense to me:

1. I farm for likes and karma

2. I start endorsing low value crap from whatever fad is trending this hour

3. Profit

On HN, I have no idea what step 2 is: what is the middle step between farming and profit that doesn't involve, like, founding a startup? What's the specific tactic on this platform?

Sorry, I didn't see the "much softer targets" remark but I disagree anyway.

Marketing on HN can be very powerful. The mindshare gain can be enormous. Niches in general are very rewarding if the underlying platform (Google/Facebook/Amazon/Ebay) doesn't deplatform you.

I don't have time to look it up but I'm sure minimaxir (Certified HN Influencer) has made a study on it.

PG remarked on it in What I've Learned from Hacker News[1]:

"But what happened to Reddit won't inevitably happen to HN. There are several local maxima. There can be places that are free for alls and places that are more thoughtful, just as there are in the real world; and people will behave differently depending on which they're in, just as they do in the real world.

I've observed this in the wild. I've seen people cross-posting on Reddit and Hacker News who actually took the trouble to write two versions, a flame for Reddit and a more subdued version for HN."

Anecdata: just today I reactivated an account on a startup I learned about from a Show HN[2]

[1] https://paulgraham.com/hackernews.html [2] https://news.ycombinator.com/item?id=24990238

Ok, thanks, this is exactly what I needed. I'm apparently too much of a casual user here because I don't even recognize minimaxir as a username. So that's gotta be the disconnect for me: all the usual tactics apply, they're just less obvious to me because I'm not engaged enough.

I appreciate you taking the time to respond thoroughly. Thanks!

Edit: it occurred to me that another potential reason that the tactics used to monetize karma farming on HN may be less obvious to me than on other platforms is because here, the tactics are more specifically designed to target me

I guess for those not sure of the context: The user Jia Tan added exploit code to the 'xz' tool as part of a larger deal. Wikipedia has a page on it here [1].

In this post, they are discussing some changes to print code specifically for the libarchive project, and some notable personalities in the security community chime in, including Colin Percival (Tarsnap among others) and Taviso (Google project zero among others).

[1] https://en.wikipedia.org/wiki/XZ_Utils_backdoor

> The user Jia Tan added exploit code to the 'xz' tool as part of a larger deal.

Various discussions on this backdoor (in rough chronological order):

* Backdoor in upstream xz/liblzma leading to SSH server compromise:† https://news.ycombinator.com/item?id=39865810

* What we know about the xz Utils backdoor that almost infected the world: https://news.ycombinator.com/item?id=39891607

* How the XZ Backdoor Works: https://news.ycombinator.com/item?id=39911311

* The xz sshd backdoor rabbithole goes quite a bit deeper: https://news.ycombinator.com/item?id=39956455

* XZ backdoor story – Initial analysis: https://news.ycombinator.com/item?id=40017310

† Original report, AFAICT.

>XZ backdoor story – Initial analysis

Here are parts 2 and 3 (weren't discussed on HN):

>Part 2: Assessing the Y, and How, of the XZ Utils incident (social engineering)

https://securelist.com/xz-backdoor-story-part-2-social-engin...

>Part 3: XZ backdoor. Hook analysis

https://securelist.com/xz-backdoor-part-3-hooking-ssh/113007...

Something tells me that somewhere deep in a millitary facility somewhere, somebody is getting court marshalled, if not downright worse (after having been found out, I mean ...)

  PS. Or some "unaffiliated" group somewhere is getting their SOF cut off ...

This was discussed at the time of the libxz revelations and they did re-review all commits in light of this: https://github.com/libarchive/libarchive/issues/2103

imho it's embarrassing that this got merged in the first place.

it's not a major flaw, and no exploit. but it seems as if nobody paid due attention to actual changes.

Funny how everyone's competent in hindsight and eager to share that.

Wild. All of these carefully inserted “seemingly innocuous” changes in the ecosystem to end up contributing to a wider exploit.

This comment sums it up nicely with a gif: https://github.com/libarchive/libarchive/pull/1609#issuecomm...

this explot is basically "death by a 1000 papercuts", now that i think about it.

More like assemble a sword from 1000 sheets of paper and stab everyone.

Am I the only one a little concerned that no obvious attack has been found from this?

It seems doubtful that a state actor is trying to use terminal escape sequences to hide an error message... The state actor wants code execution, not the ability to backspace some warning on a developers terminal. Besides, using such a vulnerability seems far too dangerous - those escape sequences would be plainly obvious in any log file or any inspection of files on-disk.

And at the same time, if you are an undercover state actor, there is no point in potentially revealing yourself by inserting some security problem that isn't exploitable.

The goal here is to submit what appears to be a sequence of innocuous changes, none of which on their own are “obvious” vulnerabilities. The truth is, we don’t know what the strategic depths of this actor are. It may be years before we know whether an attack is successful.

For example — and this is just hypothetical - the author may have found that some consumer of this codebase uses it in a script, and consumes console output in some form. By modifying its output to behave differently, they may be able to influence the consumer’s execution in some clever way so as to create other conditions necessary for additional exploitation.

Or - the PR could have just been a test to gauge the scrutiny of the approvers.

The "funny" thing here is that this is (somewhat, perhaps?) how an AI intelligent beyond human capacity might execute an attack - or what an attack by one such might feel like: Lots of apparently unrelated actions, many or all of which make no sense ...

... (until and if you see the larger picture, which might be insurmountably difficult ...

... this, coupled with AI-level scalability of social engineering, at AI-level scale -and- with an AI-level understanding of "known-outcomes" that might be desirable towards given goals: "Leader change", etc.-)

There had been code execution exploits tied to escape sequences, for example:

https://news.ycombinator.com/item?id=40428032 - Abusing url handling in iTerm2 and Hyper for code execution (2024-05-21)

I'm a little unclear as to why JiaT75's github account still exists? Surely this should be nuked from orbit so that no one accidentally ends up using their shady code?

History is useful

removing their account doesnt remove their commits.

The deletion of the account would not delete commits associated with it. The commit would still contain everything potentially malicious, plus a reference to an account that would be deleted. Which is actually worse, you cant track what code a malicious actor has contributed (easily). So the correct thing to do is take away login / deactivate the account, and then start going through all contributions and check them via the account that references all of this.

Would be cool if there was a big warning icon with "ACCOUNT LOCKED: STATE ACTOR" like for cheaters on chess websites

It's useful to keep it up so the public can scrutinize all their commits and fix anything suspicious.

Sure, but there is zero indication of that on their user page. At the very least the account should be disabled, all repos should be archived, and a big fat warning banner should be prominently visible. The current state of affairs seems irresponsible.

Labeling - bringing up the actor, and the method involved - is useful.-

Why do you write all your periods "." in this post (but not in others, per your comment history) as ".-"?

Hate to bother you with a reply on this, but, since you mention ...

I beg to differ, I actually checked.-

  I'd hate it if I had been less than consistent :)
  PS. At least from about 2021 on ...

What I found was a bunch of occurrences '...' without a hyphen, which I assumed indicated the general trend, but on further searching I see that it is as you say. Nonetheless, if I may still ask, why?

(I am in the end unsure about your question, but will try to answer it, if, by asking, you mean my general use of ".-"?

I nonetheless appreciate your curiosity).-

> … if, by asking, you mean my general use of ".-"?

Yup, that's what I mean.

It is something of a mark of style - which, since you kindly ask - I inherited from my father, and I use as a sort of homage.-

PS. It has also, countless times, saved me when I "retroactively" needed to claim authorship of something I had writen, by pointing it out. Most people either don't notice or do and don't say ...

Thank you—a nice and subtle homage.

I think it is. Also, as best I can, with my life - as do we all, don't we? Trying to be worthy of having been put here by those that came before us ...

Thanks for your attention to detail and the opportunity to converse.-

> The diff doesn't make this obvious due to the removal of a newline in a parameter list.

I like to separate every little intentional change into their own commits. So a formatting change would be separated into its own commit.

If you are looking for “red flags” notice if the diff is clean or not according to what you expect to see changed; if you only expect to see some error text change then multiple lines being changed is weird. Also use a decent diff viewer which is somewhat content/language-aware.

why wouldn’t they just immediately reject it if they know its the same guy from the xz thing?

[2021]

good thing that was in the title

Asking in seriousness: did you comment without reading the link? If so, why? I legit don't understand why people comment on things without having read them, and I would like to

Frist Psot culture is real: everyone else sees the “contribution”, and if it happens to be of marginal value, it earns karma.

Stochastic karma farming benefits from a larger N, hence posting without reading.

Thanks for weighing in! But this is just speculation, right? Are you speaking from experience? I'm specifically looking for someone who does it to explain it.

I mean, I can make my own guesses about various forms of attention seeking and hopes to somehow cash in on high karma all day long but, to me it feels like HN is among the worst possible venues for that. I'm not aware of easy ways to convert HN karma into cash flow like you maybe could with followers on other platforms. So I don't immediately see a benefit in just farming karma for its own sake.

Is there some benefit to karma farming I'm not aware of? Like, some points-to-dollars conversion stream that I'm not in the loop on?

So, hacker news internet points, yes, mostly useless.

That said, visibility in this community IS valuable.

How many billions of dollars flow through yc companies?

Obligatory discussion of hacker news karma points: https://news.ycombinator.com/item?id=35174825

I'm not who you asked, but sometimes the comments are more interesting (or perhaps intriguing, enticing, is a better way to put it) than the submission itself. Sometimes of those times my interest in the submission grows with reading some of the discussion, and then I'll read it.

(Not to say I always do this, but I do definitely click first into comments more often than I go straight for the article - it allows a much lower bar for what seems initially interesting, and I've read a lot more fascinating stuff (submissions and discussions) than I would have otherwise that way.)

Interesting! But this sounds like an explanation why you read the comments without reading the article, not why you comment without reading, and those aren't inherently the same thing.

So to clarify: do you comment on the content of the post without reading it? I'm specifically interested in why people comment on links and articles they didn't read. And for maximum clarity here, I mean commenting on the content of the article, not just contributing to the various related discussions it spawns.

And to reiterate, I'm asking in earnest. It's not something I would do, so I'd like someone who does to weigh in.

Uh, maybe. Less likely to than just reading comments for sure.

But for example this thread is on such a tangent (and it could be a hell of a lot less) that TFA is completely irrelevant to what I would comment or how it would be received, so yes I might; almost certainly have on several occasions (over years and wouldn't-like-to-think-many comments).

Too late to edit but I just wanted to add never a top-level comment, but in thread probably quite a lot actually.

And as an example of how it could be much less of a tangent, more related but still not require reading TFA to comment, I saw something recently where a top-level comment was along the lines 'I would use strace for this personally', and then the thread was all about strace. Commenting in that thread, other than to compare it to whatever OP was doing, doesn't really require reading the article, because it's about something else now, and that's been made clear from the top-level comment.

I suppose it's kind of like joining a discussion at a party - you don't need to be excluded just because you missed the thing that started the conversation. Difference here is if you want/need to or are interested, you can still just go and read it without someone having to awkwardly/hurriedly fill you in.

The PR is originally from 2021 and has been merged back then, but there's more recent discussion on it about whether this introduces a vulnerability.

Chronology. "the XZ thing" happened 3 years after merging.

@dang: Please add [2021] to the title.

Only the original commit was from 2021. The discussion is recent.

Recent is relative, the latest comment is from April 2024, not something that popped up today.

Why? It is immediately obvious that the PR was published 3 years ago. Github shows the timestamp very clearly in multiple places.

Yes, but this is not about the PR, it is about the discussion. The dates are shown very clearly in many places too. ffs

Discussion is from this year, try reading for context

It's kind of ridiculous how few lines of code can trigger such a huge and complex discussion about thread safety, where almost all developers involved actually aren't really certain whether or not this can be exploited.

I just wish people would stop writing C code for libraries that consume arbitrary data.

Yes, please stop writing C and C++ code! (... and leave the high paying jobs to us of course)

In this case it doesn't matter whether it's C or Rust. Both languages, if using their default string formatting facilities, are vulnerable.

People in the Western world are comfortable and people in China are very motivated.

I've been reading "the man who solved the market" about Jim Simons and his hedge fund Renaissance. This reminds me that there was a period just after the fall of the Soviet Union where Renaissance was flooded with very technically strong, very motivated, very hard working, and very fraudulent ex-USSR people.

They're leaking guys, wake up.

I'd be surprised if China has anything to do with this operation.

Who do you favor? One of the Russian Intel agencies? NSA? The Grugq?

At some point FOSS will need it's own agency, at this rate :)

  PS. The GNU GRU?

That is actually a thought.-

Reply on news.ycombinator.com