Hacker News Reader

Comment by returningfory2 | Hacker News Reader

A couple of years ago I wrote a Go library that wraps the xz C code and allows you to do xz compression in Go: https://github.com/jamespfennell/xz

About a week ago I received the first PR on that repo, to upgrade to 5.6.1. I thought it was odd to get such a random PR...it's not the same GitHub account as upstream though.

View on HN

As a bit of an aside, I would never accept a PR like this, and would always update $large_vendored_dependency myself. This is unreviewable, and trivial to insert any backdoor (unless you go through the motions of updating it yourself and diffing, at which point the PR becomes superfluous). I'd be wary even from a well-known author unless I knew them personally on some level (real-life or via internet). Not that I wouldn't trust them, but people's machines or accounts can get compromised, people can have psychotic episodes, things like that. At the very least I'd like to have some out-of-band "is this really you?" signal.

This is how I once inserted a joke in one of our (private) repos that would randomly send cryptic messages to our chat channel. This was pretty harmless and just a joke (there's some context that made it funny), but it took them years to find it – and that was only because I told them after I quit.

That said, looking at the GitHub account I'd be surprised if there's anything nefarious going on here. Probably just someone using your repo, seeing it's outdated, and updating it.

The (most?) popular SQLite driver for Go often gets PRs to update the SQLite C amalgamation, which the owner politely declines (and I appreciate him for that stance, and for taking on the maintenance burden it brings).

e.g., https://github.com/mattn/go-sqlite3/pull/1042#issuecomment-1...

Meanwhile SQLite itself doesn't accept any patches for anything; if you show the author one he will at best rewrite it.

In this case, the project is using Git submodules for its vendored dependencies, so you can trivially cryptographically verify that they have vendored the correct dependency just by checking the commit hash. It looks really crazy on Github but in most git clients it will just display the commit hash change.

Hey all, I’m the author of that PR. Just posted to Github with additional context: https://github.com/jamespfennell/xz/pull/2#issuecomment-2027...

That sucks to have people write mails to your employer...

To be honest, I probably wouldn't have noticed the comments on the PR if it wasn't for that since my Github notifications are an absolute mess. Thankfully, my employer has been super supportive throughout this :D

I appreciated your detailed update!

The dopamine hits from updating stuff should come to an end, it should be thought of as adding potentially new bugs or exploits, unless the update fixes a CVE. Also Github needs to remove the green colors and checkmarks in PR's to prevent these dopamine traps from overriding any critical thinking

Counterpoint: if you wait to keep things up to date until there's a CVE, there's a higher likelihood that things will break doing such a massive upgrade, and this may slow down a very time-sensitive CVE response. Allowing people to feel rewarded for keeping things up to date is not inherently a bad thing. As with all things, the balance point will vary from project to project!

Exactly. You don’t want to be bleeding edge (churn, bugs) but in general you usually don’t want to be on the oldest supported version either (let alone unsupported).

Risk/reward depends on the usecase of course. For a startup I’d be on the .1 version of the newest major version (never .0) if there are new features I want. For enterprise, probably the oldest LTS I can get away with.

I strongly disagree. If you don’t update your dependencies then it’s easy to lose the institutional knowledge of how to update them, and who actually owns that obscure area of your code base that depends on them. Then you get a real CVE and have to work out everything in a hurry.

If you have a large code base and organisation then keep doing those upgrades so it won’t be a problem when it really matters. If it’s painful, or touches too many areas of the code you’ll be forced to refactor things so that ceases to be a problem, and you might even manage to contain things so well that you can swap implementations relatively easily when needed.

I don't want to read too much into it, but the person (supposedly) submitting the PR seems to work at 1Password since December last year, as per his Linkedin. (And his Linkedin page has a link to the Github profile that made the PR).

Yeah the GitHub account looks really really legitimate. Maybe it was compromised though?

What looks legit about a gmail address and some stock art for a profile?

[Deleted per below]

Can you stay in that org after leaving Google?

whoever is in charge of removing people from the Google github org has the itchiest trigger finger in the whole exiting-the-company process tree.

You are not looking at the right profile. This is the profile that people are talking about: https://github.com/jaredallard

Oops, you're absolutely correct. Deleted (via edit) my comment above. Thanks.

He was just (50 minutes ago) removed from the oss fuzz repo.

I hope this also (at least temporarily until verification of 'bad/good') remove him from the org?

Plus the README.md that is just a rickroll

The 2 GMail accounts are 85% / mainly associated with XZ work, since 2021, per searching for them explicitly via Google.

The PR's two commits are signed by a key that was also used to sign previous commits belonging to that author.

Hold up, are you saying that https://github.com/jaredallard and the accounts affiliated with this XZ backdoor share a PGP key? Or something else?

No, this account made a PR and their commits were signed [1]. Take a look at their other repositories, e.g. they did AoC 2023 in Rust and published it, the commits in that repository are signed by the same key. So this is not (just) a GitHub account compromise.

I find this aspect to be an outlier, the other attacker accounts were cutouts. So this doesn't quite make sense to me.

[1] https://github.com/jamespfennell/xz/pull/2/commits

If I were trying to compromise supply chains, getting into someplace like 1Password would be high up on the list.

Poor guy, he's probably going to get the third degree now.

As a 1Password user, I just got rather nervous.

Yubikeys starting to look kinda yummy.

Hardware gets backdoored too, remember Crypto AG?

They're definitely a real person. I know cause that "1Password employee since December" is a person I know IRL and worked with for years at their prior employer. They're not a no-name person or a fake identity just FYI. Please don't be witch hunting; this genuinely looks like an unfortunate case where Jared was merely proactively doing their job by trying to get an externally maintained golang bindings of XZ to the latest version of XZ. Jared's pretty fantastic to work with and is definitely the type of person to be filing PRs on external tools to get them to update dependencies. I think the timing is comically bad, but I can vouch for Jared.

https://github.com/jamespfennell/xz/pull/2

> it's not the same GitHub account as upstream

This is valuable information, and a sign that this may be the tip of an iceberg.

There was also a bug report in Debian which requested updating xz-utils to 5.6.1: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1067708

That's the same Hans Jansen mentioned here: https://boehs.org/node/everything-i-know-about-the-xz-backdo...

IMO your prior on this should be that it's most likely just someone innocently updating a dependency.

The backdoor (test binary blob and autoconf) is not part of the pull request.

Suddenly anything like that becomes super suspicious.

I wonder how this will affect the OS community in general.

More caution might not be a bad thing.

Internet detectives at work in this thread!

Reply on news.ycombinator.com