Hacker News Reader

Comment by bhickey | Hacker News Reader

Some months ago after recoiling in horror from a vulnerability in rust I arrived at the grim realization that the "supply chain attack" problem is intractable in impure languages. When a transitive dependency can do anything it will inevitably do something. I don't expect to convert everyone (or anyone) to object capabilities and purity, but I do hope the experiences of the past year have put the fear of God into developers. We should aspire to build systems where a suddenly malicious or busted dependency can't knock over the whole tower.

View on HN

Author here.

You do realise that the main source of dangerous vulnerability is low level infrastructure that do not come from these package managers ? The purity and capabilities sadly solve nothing when the source of the problem is the fact that NTP, OpenSSL, OpenSSH, the BGP daemon, the libc you run on, etc are all maintained and "fixed" and evolved by people that can barely pay for their food, even less for basic tests ?

Like i love capabilities and i would love to have more, but the basic problem here is that the low level that you totally depend on for the capabilities to work cannot even be funded to work.

Hell you do realise that Autoconf has not had a maintainer since 2012? If it was pwned and had injected backdoor in your software since then, we would have noone to fix it.

nonsense.

autoconf didn't have a core developer. But it had many distro package managers.

there's nobody adding code/fixing bugs, but there are plenty people reading and looking at diffs before packaging it in distros.

then the js libs have neither core maintainers nor anybody at npm corp that cares about security or reading anything.

the top comment is correct to care about those before system libraries.

I don't agree. When there's a problem in OpenSSL, OpenSSH, or libc, nearly every company with an online presence has as freak-out and rushes to patch.

When there's a vulnerability or malware found in a python or npm package, 75% of the tech world does a quick audit (or not), shrugs, and goes on with their day.

Why the weak words? "reading", "looking" is not "understanding".

IANADev, but surely if they were going as deep as understanding patches they'd practically be a project maintainer rather than a package maintainer. Expecting package maintainers to have even reviewed diffs seems like high expectations. I wouldn't expect them to do more than the actual packaging.

Weak words seem appropriate.

I've hacked together a couple of packages for personal use but didn't even go so far at to look what language the code was in. I'm assuming a lot of packages from ppa-s are produced in an automated way with no code review, and depending on your distro that might also be true of some principal libraries.

> when the source of the problem is the fact that NTP, OpenSSL, OpenSSH, the BGP daemon, the libc you run on, etc are all maintained and "fixed" and evolved by people that can barely pay for their food, even less for basic tests ?

My take: I suspect that (perhaps a majority) of us here have a distant and abstract understanding of long-term poverty (eg:hunger on the table for years at a stretch). I'd offer that even the memory of real poverty doesn't equate with presently living in a long state of active poverty.

Translating across that gap is hard.

note: I'm not socially critiquing HN here. I am opining that survivor bias may be in play, however.

> NTP, OpenSSL, OpenSSH, the BGP daemon, the libc ... by people that can barely pay for their food

I think those used to be national or telco lab projects in 60s-80s. The values created are hard to exploit and to capitalize. Perhaps it's a policy problem.

Yeah, I think that's the most viable solution to this problem. The US federal govt should fund software projects that are widely used. It's good for our national security, and it facilitates economic growth.

This could either be grants to maintainers, grants to software nonprofits like linux foundation, or some kind of national software lab that employs people to maintain and audit open source software (or ideally, all of the above)

They would not pay non-USA citizens, and the NSA would be asking for backdoors.

OpenSSL was at various times prior to 2014 maintained by a gamer guild, and by nobody.

> are all maintained and "fixed" and evolved by people that can barely pay for their food, even less for basic tests ?

I suspect many people working on OSS are getting paid, if not handsomely, then enough to survive.

I would be surprised if they even get a thank you per 1,000 users.

I advise you to go read the Road and Bridges report i linked in the blogpost.

As of today, the reality is not at all. As I mentioned elsewhere there are zero maintainer of autoconf since 2012, even less one well paid.

Log4j devs took unpaid time off to fix it.

OpenSSL dev lived for years with less than 10k per year.

> there are zero maintainer of autoconf since 2012,

Paul's work doesn't count?

Its not intractable. You just need to take the concept of a software supply chain seriously.

E.G, every version of every dependency you use (either directly or transitivly) needs to be audited and approved by a trusted supplier. This isn't nearly as bad as it sounds. You only need 1 trusted organization across the entire industry to certify a baseline of trusted software. Maybe a second organization to cover people who don't trust the first.

The NSA, some other part of US cyber command, or a comparable agency from a different government are all natural choices; as they have a vested interest in protecting critical domestic infrastructure but no commercial interest in keeping their work proprietary. Since that infrastructure all runs the same software as the rest of industry, we all benefit.

> The NSA, some other part of US cyber command, or a comparable agency from a different government are all natural choices; as they have a vested interest in protecting critical domestic infrastructure

NSA is not a natural choice, since they are an active threat that has a history of abusing trust in their defensive mandate to insert vulnerabilities into systems in pursuit of their offensive mission.

Then you can decide for your org to not trust them and perhaps use a different certifier. i would welcome them offering such a service instead of having none.

Trust like this is transitive, so I would not trust any organization that placed such blind faith in the NSA.

I hope your brave new world includes a requirement for orgs to disclose their software auditor, so my org can avoid trusting orgs like yours.

> You only need 1 trusted organization across the entire industry to certify a baseline of trusted software.

So it's not just bad, but impossible. Got it.

More to the point, software "auditing" does not work reliably against deliberately malicious code written by competent actors, never has, and never will. It's too easy to slip in subtle back doors that reviewers won't notice, and at the same time too hard to keep up with the review workload.

I would suggest the Debian Project and similar distributions as a fairly good scaffold on which to build such a system. All it needs is a notarisation mechanism, so certify that a person has had eyes on the code and says that it is ok. Plus, I'd trust them more than the US government. Though there's no harm in having multiple bodies all certify a code-base.

But Linus has said distributions are bad because they don't let him fork libraries as he pleases!

I guess the downvotes stem from not knowing what specific episode I'm talking about

> The NSA, […has] a vested interest in protecting critical domestic infrastructure

One of the biggest cluster of cyber attacks ever, WannaCry, is attributed to NSA directly. They developed and sat on the exploit for 5 years, until it leaked, because even the NSA can be compromised. Government lives in a “we need good guys to have the biggest guns” Cold War-like mindset and are terrible role models for championing cyber security.

Auditing in general is primarily used as a tool to shift liability, or “ass-covering”, and sell a false sense of security by abusing information asymmetry. That said, there’s nothing wrong with auditing when deployed within a more informed and aligned incentive structure.

> The NSA

Seriously? As soon as the NSA certifies a piece of code it'll be dropped like a hot potato by everyone outside of the US, and probably by a large percentage of the people inside it.

If such a body exists, which seems like a reasonable idea, it has to be nongovernmental and not controlled by large corporations either.

This doesn’t make sense. The NSA has certified lots of crypto protocols that we use.

Those also presumably have been checked, verified, and certified by many other governments, institutions, and independent mathematicians.

I'm not against the NSA, FSB, 3PLA, GCHQ or any other letter based agency certifying things. I'm saying they can't become the main official certifying body for FOSS because it simply won't be accepted by the FOSS community or by other world governments.

These days it seems those protocols are used in spite of NSA involvement. See what happened with Speck in the Linux kernel. https://lwn.net/Articles/761992/

And if you has to be Russian or China government certified, all this protocols are out of question.

Ok, even if we could all agree on a single trusted auditor, that would still put us in a very bad position. You would be limited to only use that baseline trusted set of software, which I guarantee won’t be sufficient for all the things you need as software grows and develops.

You would be right if we were all running properly engineered hardware and software, but software engineering stopped being a subcategory of engineering 30+ years ago.

Software engineers are all blacksmiths, and software products equates to artisan made weapons. A food poisoning traced to a chef's knife from a workshop affects all users of the knife, but the impact is often localized to the shop and lessons learned are shared less as a scientific theory but more or less as an epic(see how "postmortems" are requested or written).

I predict that software supply chain concept will start working when software become a field of engineering, as in a single bit of error stops being the direct cause of a catastrophic failure(e.g. a web server bootloop), parts become replaceable(e.g. universal SATA host driver compiled against musl libc running just fine with glibc or proprietary Unix libc), so on and forth, maybe once anyone will be able to code or AI would start taking over us, or something - but at this moment where everyone touching software must be all manager-architect-manufacturer-user that knows where to put a chalk mark, even the idea of software "components" only barely work.

Software exists outside of USA.

I mean, I'm in Canada, friendly as it gets, and still - is NSA certifying a piece of software a good thing, from my perspective? From my company's perspective? From my Canadian sovereign government's perspective? There may be second order effects at play here.

NIST in consultations with various US public and private actors (including NSA) already determine the cryptographic algorithms that are used to run the internet. Canada is still very friendly with the US - the countries that have literally been bombed by the US - continue to use these algorithms.

Governments across the world have large gaps in their understanding of software and its relation to the world.

> to certify a baseline of trusted software.

check how it works with FIPS for USA government. Out of date and you can't use a regular distribution because it doesn't use it, because it's expensive to get certified.

the current fragility of the "software supply chain" stems from package managers and their perverse economic incentives, and this is just as true for "pure languages". Vendoring and auditing dependencies greatly reduces the attack surface area.

Can you elaborate on the incentives of package managers? The problem I see with Rust is that it's just really easy to add dependebcies in a way that is difficult in C++.

I don't follow your point about purity. A dependency in a side-effect free language can't suddenly open a socket or access the file system. The attack surface is much smaller.

> A dependency in a side-effect free language can't suddenly open a socket or access the file system.

A dependency can exploit your system deliberately, not just with side effects. If you're not strictly pinning all of your dependency versions, one malicious update that makes it to the package manager can ruin your whole world.

Our package managers operate entirely on trust. Nobody is guaranteed to be checking to make sure some random package's maintainer isn't submitting malicious updates.

As a community we're all lazy and basically hope that somebody else is invested enough to check everyone else's work. In the case of OpenSSL, we've found that some projects become so complex that almost nobody does for long periods of time.

I think the thing that makes it more tractable in pure languages is that you at least get the chance to control the dependency's inputs and inspect all of its outputs, so it's theoretically possible to limit what it can do to you (depending on how you are using those outputs). I'm not saying it's usually practical, but it's at least conceivable. And there should be at least a few cases where it's actually practical.

In an impure language, it can do any random thing to any random part of the whole system and you won't even get the chance to know what it is doing.

Are there any truly pure package management systems out there, though?

Haskell has Safe Haskell, which more or less guarantees that the actual Haskell code compiled through the normal path is pure (but not that the code is total, and total under resource constraints, which you also need). The build system itself is not pure, though, so you have no guarantee that a dependency's build scripts don't put Evil Things into your program outside of the actual language framework.

Any others?

It sounds like Safe Haskell might be getting deprecated soon [0]; it sounds like it isn't used much in practice...

[0]: https://discourse.haskell.org/t/deprecating-safe-haskell-or-...

> not just with side effects

I think you're using "side effects" in a colloquial sense while I'm using it in a PL sense.

Pure code is deterministic, operating only on its params. Impure code—code with side effects—is something that depends on state outside of its parameters.

> Our package managers operate entirely on trust

Cannot agree with this oversimplification.

Exists community. Yes, every human have his own motives, but many live in countries with effective law enforcement, and many people obey Western Values.

Also important, software is a chain with many parts, some parts of one chain could make really large disruptions, some are less significant.

So overall, real formula includes trust, humanity values, laws, government effectiveness and inertia of this chain.

Rust : Cargo :: Haskell : Hackage . It's extremely easy to add dependencies and that problem is independent of purity.

As an example of an "impure language" with security in the form you envision, Deno allows you to specify access controls on a per-script or per-dependency basis. You can specify imports that prevent dependencies from opening sockets or writing files or reading files.

Many years ago npm inc treated download and package counts as KPI. They pushed back against sensible measures like code signing, instead centralizing the ecosystem and encouraging cross-dependencies within their registry. A more security-oriented package manager would have focused on code signing and vendoring strategies.

Haskell is largely pure. There's nothing in place to stop a dependency from putting unsafePerformIO on blast. If a dependency can only do pure functional computation then the worst things it can do are like (1+1=4) or diverge.

From your description it sounds like Deno is using object capabilities -- that's a great step forward.

> Many years ago npm inc treated download and package counts as KPI.

That's bonkers. I had no idea.

> There's nothing in place to stop a dependency from putting unsafePerformIO on blast.

     import safe MyDependency

... but do you have to make sure that nothing in the tree is claiming to be Trustworthy. Which is actually pretty easy.

I believe GP misspoke; to my knowledge Deno does not allow you to set permissions on dependencies. It's not using object capabilities as far as I'm aware. I wish it were!

https://github.com/denoland/deno/issues/171

Functional purity is the wrong abstraction to ask for, and doesn't protect you as such (consider a Haskell library that, very purely, asks to write to ~/.bashrc).

Sandboxing modules is what you want, and that can be done with impure languages too. Sure, pure functional programming might make it easier, but as others pointed out, Safe Haskell and friends are very very far from mainstream.

Meanwhile, Firefox compiles some C/C++ dependencies into wasm, and runs them in sandboxes.

Can you give some examples of such a pure language? Haskell is definitely not it, since you can always call unsafePerformIO anywhere to do anything. I would guess Idris has something similar, but maybe I'm wrong.

Also, if a dependency actually needs IO (either a networking library, or a DB library, or just for logging purposes), the guarantees you're talking about again go out the window.

And, moving to the more extreme scale, you can never be 100% sure that some dependency hasn't found a way to trick the compiler or runtime system into doing something it's not supposed to be possible to do, even if it consists of (ostensibly) pure code only.

> Can you give some examples of such a pure language?

BPF and eBPF, though this feels unsatisfying.

Hence why I check in vendor dependencies. It is wild to me that people pull fresh their dependencies on builds, trusting patch versions that really should have been major versions.

Vendoring is an option, but if you place some trust in your registry a lockfile is totally fine.

Reply on news.ycombinator.com