Comment by jacquesm | Hacker News Reader

Comment by jacquesm | Hacker News Reader

Parent Full thread

The OS can't help as long as users can create files. But eco-systems (whether Rust, Go, Node or Python doesn't really matter) can help by managing their name spaces properly. The thing that went wrong here is that someone can silently replace a package in a namespace that you think is unique by overruling it through a namespace that takes precedence. Besides the obvious usability issues (this is not the kind of functionality that you would want in any circumstances) if a package suddenly gets pulled from an entirely different repository compared to where the previous instance came from and does so silently (without throwing an error and without so much as a pause to ask the user whether this really is what they want) then I really don't see what kind of OS level awareness of the world at large would be required to fix that.

The problem does not stem from the operating system level, and so it should not be fixed there, and it likely can't be fixed there, at least not without possibly breaking a whole bunch of stuff that is working just fine today.

As explained in another thread, nothing went wrong with the tooling and namespacing. The PyTorch devs misused a feature intended for a different use-case.

I took a look and the vulnerability that was exploited is warned about and described in the documentation.

> Warning > Using this option to search for packages which are not in the main repository (such as private packages) is unsafe, per a security vulnerability called dependency confusion: an attacker can claim the package on the public repository in a way that will ensure it gets chosen over the private package.

https://pip.pypa.io/en/stable/cli/pip_install/#examples

And that's even further removed from the claim that this could be fixed at the operating system level: an operating system is powerless against people that bypass security measures willfully. But regardless, this option, even if available should always result in a warning when there is a change of effective source for a chunk of code that was previously included blindly from another source.

I agree the issue is not caused by OS; rather my point is, OS is a convenient layer to prevent it.

You mention some mechanisms for preventing these issues at packaging level. But so long as bad actors can sneak malicious commits, it's an open problem. Here a whole package was replaced, in another scenario a single line of malicious code can be sneaked in, released, rubber stamped and shipped via bona fide channels. Ensuring no single bad commit is made in any package or its dependency doesn't seem like a realistic task.

As other posters suggest, this could be somewhat mitigated by mobile-style finer-grained permissions, even if the problem doesn't stem from an OS failure per se.

Those who want those protections can effectively already have them, but convenience (and modern CI/CD tooling) dictates that they will either be disabled or ignored completely.

If you try to fix a problem at the wrong layer all you will end up doing is create a bunch of false positives which will result in those protections being disabled. Then, by the time you need them they will no longer be active. A problem like this should be dealt with as close to the source as possible. A defense-in-depth strategy is a sound one but only if it does not add burdens that will lead to the defenses being disarmed in peaceful times.

Finally, I fail to see which finer grained permission would have stopped this, though maybe someone can make some suggestions. Not allowing the use of Node, Python, Go or Rust would seem to be a bit harsh and nothing short of that would seem to solve the problem without the OS having to gain massive awareness of what is going on at the application level.

Microsoft effectively fell for this by dragging everything and the kitchensink into the operating system layer, the only result of that was an even larger attack surface at a level where it mattered even more.

This is not an easy problem to solve. For starters you would have to review the way all of these dependency resolution mechanisms work and flag the ones that you think contain risk. Then you need to get the suppliers of those systems - and others like it - to agree on a standardized way of vetting the import process and finally you would have to make the OS aware of a set of signatures or certificates that would stop such an attack in its tracks.

The alternatives are doing away with easy dependency resolution or taking charge of the root cause and fixing it: the admission that decentralized source code control is a great thing to have but unsuitable for the distribution of packages for immediate introduction to production systems based on the say so of whoever controls the namespaces. The kind of centralized control that would require is something that the likes of Apple and Microsoft can provide, no Linux distribution is dominant enough to be able to solve this (nor are they wealthy enough). Distros have a hard enough time vetting their own output.

For example, blanket access to filesystem seems bad. Start with, most apps get access to some particular subdir of $HOME, like say "Documents", their own config for and systemwide files (not user-specific). Then you have a directory of home which you know is more vulnerable, but the rest of it, like .ssh, is protected. Select programs like ssh or rsync can have access.

Perhaps a simpler starting point is a runner script that enforces such rules on an opt-in basis from the user, and tooling that makes it easy to run binaries through this. Like a reverse sudo sort of thing.

I'm certainly not saying it's easy, but I'm not sure it is impossible either. It's a bit like an internal firewall.

Even if apps request blanket permissions, that's a win already. On mobile, when apps do it, it is a big red flag for me and I skip.

All that was required here was a very noticeable warning that a dependency that was previously sourced from one place now came from another. Doubly so, because, as a commenter further up points out, the new source is outside of the private repository and the risk of confusion was well known.

Reply on news.ycombinator.com