Hacker News Reader

Comment by globular-toast | Hacker News Reader

The trouble is these tools all do different things and aren't really comparable. I wouldn't even include Docker in this kind of thing as it doesn't really do anything on its own.

For me, there are two main choices today:

* An ensemble of single-purpose tools: pip, venv, pip-tools, setuptools, twine, tox,

* An all-in-one tool, for example Poetry, Pipenv or Anaconda (or Miniconda).

I prefer the former approach, but if I had to choose an all-in-one tool it would be poetry.

View on HN

Besides the dependency management, another major problem of Python is the deployment. Although Docker is not a dependency management tool, it can be used as a deployment tool which encapsulates the application to be deployed, python runtime and all other shared libraries dependencies alongside the configuration. Another deployment tool that is worth mentioning is Pyinstaller. It can pack the python runtime, the application and all dependencies into a single native executable. Pyinstaller is better for Desktop applications and for building single file executable applications.

Docker is an invaluable tool, but it's a general purpose one. It's good for Python because it's good for everything.

I have used Pyinstaller. It's good but it's a bit too magical for my tastes.

Unless you’re on macOS where you have to manually edit the pyinstaller package to comment out or point to an updated Ntlk data dump.

Well docker locally can be used an expensive isolation tool. In some sense it plays the same role as a venv.

I wouldn't use it over a clean venv management, but I have seen people who prefer using docker containers because they find venvs to not be clean/elegant.

I agree with you that Docker should not be there, but the reality is that people us it to replace some other tools (like venv).

I wonder why you prefer the former approach.

On the contrary, thanks for having included Docker in that list. It's the obvious answer to so many problems (developing, running and deploying apps, replicating deterministic Python environments, not installing linux dependencies required by Python packages directly on your machine, and so on).

BTW, to comment one of the point you made in the article, it's not that hard to run CUDA inside a container. It's less straightforward but quite well documented. You basically need nvidia-docker [1] on the host and start your containers with the runtime 'nvidia'. docker-compose still doesn't support it officially but there are workarounds. [2] I'm running it on ~50 instances in production and automated all the setup with ansible successfully.

[1] https://github.com/NVIDIA/nvidia-docker

[2] https://github.com/docker/compose/issues/6691

Why thank him for including Docker if it's the "obvious answer"? You're already using it, that's great.

Docker doesn't do anything Python specific on its own. It can be part of a pipeline but only with support from the Python specific tools which is what should be discussed in this kind of article.

The former approach is more like the Unix way: each tool does one thing and does it well. I prefer that because I can then assemble a workflow that works for me. It's easier to build pipelines when you can drop in each piece one by one.

All-in-one tools almost never do things exactly the way you want. They have a higher barrier to entry as well as a stronger lock in effect than smaller tools. If I fall out of love with venv, I can replace it with Docker. I can't just do that with an all-in-one tool.

Having said that, poetry is quite well designed and I do encourage junior developers to explore it for themselves instead of just doing what I do. If I was a junior developer today I might be quite glad for a single all-in-one tool that gets me on my feet with good practices from day one.

I think docker is great for providing isolated environment, venv has similar goals. pip-tools + docker is powerful combination, but the article doesnt mention pip-tools for some reason.

Reply on news.ycombinator.com