Hunting for Malicious Packages on PyPI
jordan-wright.com
jordan-wright.com
If you're looking for a tl;dr you can find one on Twitter (with pictures!) [0]
This research was a blast to do, and I learned a ton. Happy to answer questions!
> This is a great approach to detecting malicious code execution in Python packages.
> ... anyone want to fund making this part of @pypi?
https://twitter.com/di_codes/status/1327121326734241797
I think this is an obvious place that someone in the ecosystem could apply money and make their supply chain (and everyone else's) safer.
Most of the infrastructure on the PyPI side is in place[2], but the current checks are mostly proofs of concept/exercises of the new APIs to ensure that they don't atrophy.
[1]: https://discuss.python.org/t/what-methods-should-we-implemen...
[2]: https://github.com/pypa/warehouse/tree/master/warehouse/malw...
I'm a big believer that functions like this should be centralized under a foundation like that, and have really close connections to package manager maintainers so that we can work together towards solving the problem.
This seems like a ripe angle for package take-over.
Is there a build system out there that doesn't have this feature? Pip is both a package manager and build system since many packages are compiled at install time.
I thought pip did not run code when installing .whl files? A —-wheels-only option would then be useful.
If you use `--only-binary :all:`, that is equivalent to --wheels-only. Though not all packages ship wheels
The technique of observing syscalls has clear benefits. However might there be ways of evading this simply by setting up a some kind of delayed process so the syscall doesn't happen during the observation window or is only triggered rarely or on certain combinations that might not typically be tested (meaning it could still be caught in theory but the chances are much lower)?
1) The “observable window” is the entire installation time. If they make installs take forever, that’ll affect everyone which should raise alarms pretty quick.
2) The conditional execution is possible but the installation is done using a vanilla alpine container which will match many legitimate hosts too. And any fingerprinting activities that involve syscalls would be detected in the process.
All this to say, there’s always room to continue raising the bar!
On a dev machine though... diabolical. That might show up in a syscall, but perhaps not obviously enough that it sets off alarms.
That's the thing. If we're watching syscalls, we see these checks. These would be things like attempted file-reads. Would they be enough to set off alarms? Maybe, maybe not.
This is generally the cat/mouse game of malware detonation in general. There are attempts to make sandboxes appear realistic, but I'd argue that our use case is even simpler since running commands or making network connections during installation is not a normal thing. It might be benign, but it's abnormal enough to warrant investigation.
There will always be ways to try and get around the system, but I'm pretty firm that this will significantly raise the bar which is a Good Thing.