Verified curl
daniel.haxx.se
daniel.haxx.se
The section following then mentions signing the pre-processed source code, which I think is the wrong approach. It makes a difficult situation because of how strongly some people encourage signed source code, yet I think autotools is part of the build process and should run on the build server (and double checked by reproducible builds). If people pre-process the .orig.tar.xz they upload to Debian, this pre-processing won't be covered by reproducible builds because it happens undocumented.
The patch for "reproducible tarballs" is quite involved[0] and has rookie mistakes like "pin a specific container image using `@sha256:...` syntax, but then invoke `apt-get update` and `apt-get install` to install whatever Debian ships at that time".
Best to containerize workflows like this with hash-locked deps so they can be easily verified by others even far in the future with any OCI compatible toolchain.
GitHub had a bug early last year[0] that broke that determinism and it caused a huge uproar. So through a mixture of promises to individual projects and just so many projects relying on it, GitHub's git archive has been ossified into being deterministic (unless they want to lose a lot of goodwill among developers).
Source releases shouldn't be pre-processed in any way. They should build whatever they need to on-the-fly as part of the build.
Other project anti-patterns:
0. Maintaining creaky, fragile, old forks of other projects as dependencies.
1. Git submodules.
2. Too many dependencies.
3. Extremely large code bases that don't prune poor patterns, over-engineering, and dead features.
4. Lots of features. OpenSSL, I'm looking at you.
5. Not signing releases with a passphrase split between multiple developers to requires 2-person keying.
Package management/distro/builder/packager anti-patterns:
6. Patching the hell out of a project without pushing fixes upstream.
7. Inability or failure to source upstream from multiple independent sources, compare them, and verify chain/web of trust using cryptographic signatures.
8. Not following reproducible build guidelines.
9. Not using build caching like sccache.
10. Not building from reproducible sources periodically on the client-side to verify binaries are identical to those built by others.
11. Dependency hell of traditional (non-nix) packages importing zillions of packages.
In the past, I have occasionally downloaded a tarball from one mirror, and verified against a checksum from a different mirror (or from the official website). Back when release announcements were primarily made on mailing lists, using a mailing list archive to get a copy of the "real" checksum was also a possibility.
I definitely remember being advised to get the checksum from a different source than the tarball, a number of different times.
> Our policy says that if a contrition is good:
* contribution
Ultimately this is why we also sign the checksum.
(Now you have to protect your signature verification supply chain. It never ends)
> A (to me) surprisingly large amount of contributions are done by people who do not state a full real name
Typical good developers are not Rambo. When law enforcers come to them and force them at gunpoint to make them commit/add a new maintainer, they should not expect active resistance. Minor reminder: curl is not just some http library, they maintain their own CA list[2]. They don't need any intricate hidden lines for backdoor, CA list is a backdoor on its own.
How so? He has a rather extensive autobiography page[1] with his full name (Daniel Stenberg), plus over a dozen photos of himself[2] (and videos), and his git commits are under his full name and email address (hidden on github.com web interface just like on all repos, however). Perhaps you mistook his handle of "badger" for a pseudonym?
> he personally had legal issues with US in the past, the reasons for which were never disclosed
You thought his blog posts about it[3] were insufficiently descriptive? Or you blame him somehow for the fact that the US government never explained itself?
[1] https://daniel.haxx.se/about.html
[2] https://daniel.haxx.se/photos.html
[3] https://daniel.haxx.se/blog/2018/07/28/administrative-purgat...
Let me explain by comparing curl with XZ (and it was not me who started this comparison). XZ was initially released in 2009 by Lasse Collin. But how would you know that "Lasse Collin" actually exists? Is it a single person or a group? Maybe some papers? (unlikely, for papers see Igor Pavlov's dissertation in LZMA[1]). Do we know his country? No, all that we know that Lasse was in some "hotel with limited internet connection" at the time of the incident (hello to Russian and Chinese prisons). Who would benefit from having an autobiography page, which nowadays can be made up in 5 seconds in ChatGPT?
> Or you blame him somehow for the fact that the US government never explained itself?
Not for that. In his "Administrative Purgatory" post he makes some assumptions on reasons of visa denial. But the whole picture he showed there is that he is just a dutiful developer of a popular opensource library. Also he owns a domain. And that was enough for the US to equate him with criminals. Then why didn't he take a lesson from this that in order to remain free (this applies to traveling as well), it's acceptable to separate hobbies for information security, cryptocurrency, export-restricted technologies like HPC, AI and quantum computing and other legally complex stuff under a separate profile?
Pseudonymity in open-source (as in any other charity) is long overdue to be accepted as the norm. Let's stop building security theater on autobiographies from other people's personal pages.
[1] https://primo.nlr.ru/primo-explore/fulldisplay?vid=07NLR_VU1...
So now you're saying that the fact that (scare-quotes) Daniel Stenberg has a public autobiography page and photos and gives video-recorded in public makes you less convinced he's a real human individual?
OK, sure, you can never be sure that your neighbor or coworker or Linus Torvalds is not a CIA or FSB agent. But you are basing this on nothing.
release: https://hub.docker.com/layers/stagex/curl/latest/images/sha2...
source: https://codeberg.org/stagex/stagex/src/branch/main/packages/...
signatures: https://codeberg.org/stagex/stagex/src/branch/main/signature...
Every package is bootstrapped all the way up from a heavily reproduced 256 byte assembly seed (Stage0/live-bootstrap) and built by two or more maintainers with confirmed matching hashes, and with signatures from well known keys.
100% of commits in our repos are also signed, and every PR merge also comes with a signed merge commit by the reviewer.
Information on how to verify our keys is found in the maintainers file: https://codeberg.org/stagex/stagex/src/branch/main/MAINTAINE...
If the curl team wants a similar level of supply chain security for their own official binaries or Dockerfile we would suggest cloning our Containerfile and hash-locking all dependencies to the latest stagex release (please by all means reproduce, verify, and sign that too!).
This should be easy for curl maintainers to build and get identical hashes for their own release binaries to the ones we build and sign.
Stagex could also be used to produce source tarballs with generated files from similarly deterministic/multi-signed versions of autotools etc.
I am not convinced there is a good case for having any auto-generated files in the source archives though. Force distros to bring their own autotools, etc., imo.
Every other element on the list can be attacked if the maintainer themselves is the malicious party carrying out the attack and it is being performed with the level of sophistication in the xz attack.
So ya, I'm not saying it's security theater in all contexts, but if the context is the attack vector used in the xz attack, it's security theater.
Most everyone has a love for money, their own health, or people they wish to protect. This weakness will be exploited by those that understand a tampered version of curl in the right cryptocurrency company CI job can net billions of dollars.
We absolutely must never trust any single person in a widely relied upon software supply chain for their own safety and ours.
Any trust in any single human or machine is a very real security vulnerability.
Even without jumping to coercion, people let personal email domain names expire, they get phished, their workstation is compromised by a new steam game or IDE plugin with an RCE, etc.
I have performed and been read in on a number of attacks like this in the wild as part of pentests. It is often brutally easy.
It's not clear (to me) that Lasse knew that Jia Tan was pseudonymous. I agree with him that it's going to be harder to target curl though, as it's a larger project (maintainer-wise) and he's personally releasing all of the tarballs.
Not saying attackers would fail with curl in general. Just repeating the exact method from xz is not possible.
Why would someone state their full real name on the internet to contribute to curl? If someone wants to boost their CV, they can just write "curl, Linux kernel, GCC contributor" and provide the link to my GitHub profile upon request. Yes, someone in HR will (gasp) learn that orthoxerox is actually called Boris Kozlov in meatspace, but there's no need to broadcast this information.
apt install xyz
a python requirements.txt as well. (which is why we use Poetry).
It's frankly amazing how deeply fragmented and backwards-looking C is that this continues to be a problem. In the last 30 years, a countless number of C build systems have been created, and yet an ugly pile of obscure macros outlives them all.
We should absolutely celebrate and encourage supply chain integrity transparency writeups like this from maintainers.
I imagine the total number of trusted packages would be substantially higher.