PyPI Migrated to New Infrastructure | Hacker News Reader

PyPI Migrated to New Infrastructure

mail.python.org

52 pointsdonaldstufft18 comments

Well, great news for pypi. As for using salt -- I thought: "Well, I guess it's time I had another look, they probably have their security story together now, after all.".

So I have a look at the "Masterless Quickstart" and "Installation" pages [1,2] and find:

   wget -O - http://bootstrap.saltstack.org | sudo sh

And the server answering on 443 on that host doesn't have a certificate for that domain, or the bootstrap script (clearly it's not configured for a ssl vhost by that that name/id -- through SNI or otherwise).

So, just run some code downloaded over plain http as root on all my servers? Great plan!

edit: I think I'll stick with Ansible for now.

edit2: For those wanting a look at the salt states, from the thread[3]:

> Where are the states stored?

https://github.com/python/pypi-salt

1: http://docs.saltstack.com/topics/tutorials/quickstart.html

2: http://docs.saltstack.com/topics/installation/index.html

3: https://mail.python.org/pipermail/distutils-sig/2014-January...

On the Installation page you linked to it clearly mentions:

PLATFORM-SPECIFIC INSTALLATION INSTRUCTIONS

These guides go into detail how to install salt on a given platform.

A QuickStart is just that. Something which serves its purpose as a QuickStart. If you have security concerns then read the source code or test it out in a VM.

It doesn't change the fact that running code that's downloaded over http,unverified by any kind of signature, as root -- is a very bad idea, and just having it there in official documentation for package whose purpose is managing systems gives me a very poor impression of the projects security goals.

The alternative is:

1. get tarball

2. do full code audit

3. run code

However, almost no one will do step 2, so running code piped from wget is effectively equivalent with doing step 1 and 3. https provides some level of security, but a determined attacker can make sure you don't get warnings from their MITM attack, which gets us back to having to do step 2.

No. The alternative is to provide a gpg signature of the shell script, and instruct new users to verify the signature, before running the script. The script can then embed signature checks for any downloads.

The point isn't to make sure "everything is secure", the point is to avoid terrible advice on the fourth line of the documentation.

Another (IMHO not as good) alternative is to simply distribute the script over ssl.

A third alternative is for the instructions to show how to first get the file, then check that it is the right file -- but that implies that the reader is qualified to audit the shell script. That's a lot easier said than done. A signature would go a long way to allow the person installing to simply trust the salt project -- without a signature or an ssl distribution point, it doesn't help to trust the salt project: you have no idea what is being piped into the root shell.

And it doesn't fill me with confidence when the project advocates doing this -- that leads me to not trust the rest of the code as much, either.

As other has pointed out -- distribution via apt etc is also good -- as indicated in the platform specific instructions -- but the documentation lists the apt-platforms among those where the script has been tested.

Yes SSL or GPG is nice and provides some level of security, but think of the more fundamental problem: you're installing new software from a group of people you probably don't know, it's going to be a leap of faith either way. What you talk about are technical measures that in no way address the social problem of establishing trust in the people making those signatures.

You're missing my point.

Even if I trust saltstack, I can't trust whatever comes down over an http link. If they'd signed it (with gpg or ssl) -- then I could choose to trust the code. I'm more than ready (perhaps foolishly) to trust the code from some random group of people -- but I need to be able to trust that code -- not whatever comes down off a possibly spoofed http link.

Without signatures of some kind, there can't be any trust. It doesn't matter if I trust Debian to release a good distribution if I have no way to know if the software I'm installing is from Debian.

How to install ansible: sudo rm -rf /

Oh no! Now you can't use ansible either, because insecure installation instructions exist, and thus you are compelled to use them instead of the sensible alternatives :P

They are using salt to configure their infrastructure now instead of chef, interesting!

https://github.com/saltstack/salt

+1 for Salt. I really like it. I came to it not as a replacement of puppet or chef or other mature solutions but as a better "log in with ssh, run commands and dump some some config scripts" system. Ansible was another contender but Windows support in Salt won over (and Salt now also has ssh-only mode I hear).

Being a Pythonista and working with Chef for the last year I think that this is makes some sense. We've been recently contemplating the same migration.

Of course, this case the number of servers being managed is low enough that there isn't really a going to be much of a performance difference between Chef or Salt so this is likely just the preference of the current maintainers.

Their reason for using Salt (from the Distutils mailing list [1]):

"> Can you say a little about the choice to use Salt instead of Chef? I don't really care either way, but am just curious. Is it because Salt is written in Python, or were there other reasons (functionality, etc)?

"I’d need to ask Ernest to be sure, but I believe it was mostly that he was more familiar with it. The fact that it was written in Python was a bonus as well ;) I don’t think that there was anything that Chef was missing or that Salt had over Chef, just familiarity of the person who did most of the work. I’ll double check with Ernest to make sure there wasn’t another reason :)"

[1] https://mail.python.org/pipermail/distutils-sig/2014-January...

I have actually now asked Ernest, and that was basically the reason. Familiarity. He did most of the work and was more familiar so it enabled him to complete the work quicker. We still do use Chef on the python.org infra, just not for PyPI itself.

They felt dirty using a ruby project.

Ansible would be another awesome python solution to use in this case.

as a solo developer doing my own devops, ansible has been fantastic for its near-zero infrastructure overhead.

Why you have the notion that they despite ruby?

I'm impressed how little infrastructure they where make due with previously. PyPI hasn't been too bad considering the backend.

Reply on news.ycombinator.com