PyPI has completed its first security audit | Hacker News Reader

PyPI has completed its first security audit

blog.pypi.org

137 pointsmiketheman22 comments

Link to the report: https://github.com/trailofbits/publications/blob/master/revi...

They seem to not have analysed client-side of PIP itself, but I suppose there isn't anything you could say that isn't already obvious to everyone.

PyPI and pip are both under the "umbrella" of PyPA, but they're separate projects with (largely) separate maintainers. The audit was only scoped to the former, not the latter.

(FWIW, I don't think the security posture of pip is obvious to everyone[1], and I do think it would benefit from a separate audit!)

[1]: https://yossarian.net/res/pub/hushcon-west-2022.pdf

Interesting slides! Thanks!

`pip download --no-deps` allowing arbitrary code-execution is non-obvious, and IMO broken.

Even pip install allowing arbitrary code-execution is non-obvious, although perhaps not entirely broken.

Does it matter if the code-execution happens at `pip install` or `python myapp.py`? Using 3rd party libraries inevitably means you're allowing code-execution to 3rd parties, that's the point after all.

Yes, because you could in theory run `pip install`, then manually read through every file you've just downloaded, then run `python myapp.py`.

But every package manager seems to grant RCE to every installed package. I agree it's broken.

> then manually read through every file you've just downloaded

pip download?

Which can also execute arbitrary code according to the slides above.

> Yes, because you could in theory run `pip install`, then manually read through every file you've just downloaded, then run `python myapp.py`.

This security model is utter nonsense because no one does this.

Replace "manually read through every file" with "run your security code scanner against every file" and it becomes less nonsense, but just as applicable.

In reality this really isn't how code scans are done, so it's still a little silly, but I could theoretically see something like this being a desire.

It becomes more applicable, not just as applicable.

Amazon asked me to and I actually did it for all the Brazil third party imports...

granted it wasn't the most thorough of reviews, as is the nature with huge PRs

You're not being imaginative enough.

Evil Joe: Can you install this package in the system's python install? All users in the lab need it.

Naive Joe: Hm... Seems harmless enough enough. Let me just install locally and check if there aren't any setuid binaries in there

naivjoe:~ $ pip install --local getpwned

... checks all installed binaries look good ...

Naive Joe: Funny package name

naivjoe:~ $ sudo pip install getpwned

Naive Joe: Done!

Evil Joe: Thanks! evil laugh

Naive Joe: uh what's so funny?

Evil Joe: Nothing.

Careless, amateurish? Maybe. Obvious? Maybe not.

My understanding reading the report is that the audit is for PyPI code and infrastructure itself and not the packages it hosts. Am I right?

Yes, that's correct.

Congrats! Thanks for trailblazing and being transparent to help other central registries follow.

How much does an audit cost?

It's a bit like asking how much does a vacation cost. It rather depends where you're going and what you're doing.

I'd guess high five figures or maybe low six figures?

Five and six figure vacation costs are why so many security audits are staycations, working from home.

Depends widely on scope, complexity, client and consultancy. Example from Trail of Bits regarding blockchain audits:

    +---------+---------------------------+-------------------------+----------------------+
    | Size    | Small                     | Medium                  | Large                |
    +---------+---------------------------+-------------------------+----------------------+
    | Project | ERCs (20, 71, 4626, ...)  | Standalone arithmetic   | AMM or lending       |
    |         |                           | lib                     | protocol             |
    +---------+---------------------------+-------------------------+----------------------+
    | Pricing | $25k                      | $25-50k                 | $50-100k             |
    +---------+---------------------------+-------------------------+----------------------+
    | Timeline| 1 week                    | 1-2 weeks               | 2-4 weeks            |
    +---------+---------------------------+-------------------------+----------------------+

https://www.trailofbits.com/services/software-assurance/

Good to know. But how often are they going to do it ? Is it going to be an annual event from now on ?

Depends on funding. Ask your employer to pitch in!

Reply on news.ycombinator.com