I am not into Python, but the article lost me before it even started: you are required to install a bunch of compiler tools on your device to be able to proceed.
Excuse me? Is this really what hypermodern Python looks like?
This article lists a set of bleeding-edge tools, should you choose to add them and learn them all in one place. I wouldn't use half the tools (they're too hypermodern), but it's helpful to know where things are moving.
I also generally don't lock versions on dev machines; code should use the core, supported API, and not break on bleeding-edge functionality and API changes. I lock version on deploy machines, obviously.
Smart people on my teams don't like this approach, though, so I could be wrong.
But you can do Python this way. And beginners definitely should start by doing Python this way.
When you say beginniners, I think it depends on whether you're referring to programming neophytes in general or professional developers who are new to Python specifically. In the latter case, I actually think it's really important for newcomers to Python to get into best practices like this very early on -- indeed, pretty much immediately. Otherwise they're going to end up either being unable to use any interesting dependencies or being unable to distribute their work in a way that is easy and convenient for others to hack on. Do this with a bunch of people simultaneously and it's a big problem.
1) There's a world of difference between that and docker, and especially docker with containers for not just postgresql, but a half-dozen specialized data stores, queuing systems, MTAs, etc.
2) There's also a world of difference between having numpy / pandas / etc. in your requirements.txt, and having those pinned to a specific version. I'm okay with one or two pinned dependencies on any specific project (for example, if there's an overall project built on Django).
But if you're using the corners of standard libraries in ways where version 1.65 works and 1.73 doesn't, you're probably doing something wrong. You're probably using features which are too bleeding-edge. I'm okay with a few conditionals in code too (if library is 1.65, do X, and if it's 1.73, do Y).
When I've seen systems that to depend on nuances of specific versions, upgrades turn into "migration to [library] 1.73" and eat up weeks of developer time. It gets worse when you have cascades (upgrading library X means upgrading Y, etc.).
And goodness help you if you want to integrate two systems built in docker with pinned everything and fine-grained dependencies.
A lot of this also comes back to willing and able to say "no" to features which take 15 minutes to introduce, but cost time down the line to maintain.
Systems which install on Ubuntu without virtualenv or pip (just apt-get installing packages) are an ideal I strive for. It's usually one I don't hit (and it's also not how I develop, obviously -- it's not for me, but for my users, as well as for the discipline).
Your argument probably would be that the dependencies used should be so simple and core that the risk of it not working with someone else’s package set should be minimal or zero. That’s just a bit too extreme for my taste. I want builds to be 100% reproducible. This is exactly what modern build tools for other languages do.
Re: Docker, I don’t think anyone is claiming pip and virtualenv are somehow a replacement for that.
Re: apt-get, we tend to actually avoid this. It’s really not a good package manager at all and can easily break. We’re going in the direction of nix instead and may even port our entire Python workflow over to it or bazel at some point.
(1) I want builds to be 100% reproducible on deployment servers and on CI/CD pipelines. Otherwise, you can undebuggable Heisenbugs. On the other hand, I don't want builds to be reproducible between developer machines. If I'm running Python 3.6 on Ubuntu, and another developer is running Python 3.7 on a Mac, and we have slightly different versions of numpy, that makes sure the system is not too brittle. Come to think of it, if I had infinite resources, I'd have several build machines with different (reproducible) configurations.
(2) I'm a lot more spartan about dependencies than other developers I've met.
(3) I'd never use apt to manage Python packages myself in something I'm working on. The constraint is in the other direction. If I build a tool, a user ought to be able to install it using apt in some future version of Debian, and likewise for other systems. Even if that's an abstract user.
I've found that if I develop this way, the upsides outweigh the downsides, especially over extended periods. A lot of software gets built like a system which can only live in one places. There's a set of AWS machines, code on them, and that's the system. There might be a few copies of it (stage+dev+etc.), but you can't move it somewhere else. I like systems I build to be portable. Someone can bring them up-and-running on their own machine, ideally in a few minutes. I've always found that to be cheaper, in the long term.
On projects I've worked on before, I think this would have made sense /technically/, given project priorities, but so did many other things which weren't done. It's a lot easier to make the case for resources for customer-facing features than for technical debt or infrastructure. So there's the political component too, which varies organization-by-organization.
This is already done in a lot of projects with hardware. The Linux kernel will run on a thousand hardware and software configurations before integrating features.
If I did this, I'd probably want at least three builds:
* my pinned deployment versions (sometimes a release or two behind, sometimes bleeding-edge)
* latest released version; and
* HEAD
If an upstream project is introducing a breaking change, I'd know immediately. That'd be super-helpful, probably both to me and to those projects.
Come to think of it, the right way to do this might be to have three virtualenvs on my local machine, rather than just different targets in CI/CD....
Usually distributions bundle one or two specific versions of Python. Pyenv makes it super easy to install and use all the versions of Python that you want.
Even though for most people it might be enough to just use whatever version of Python comes installed with your system, for a team it might be important that everyone has the exact same version.
Moreover, pyenv-virtualenv makes it painless to use virtualenvs and so I recommend you give pyenv a try even if you do not need additional Python versions.
Why not just install the required version of Python, maybe from a 3rd party repo if not available in the main repos? Why would I ever want to get a compiler toolchain to get my Python interpreter?
That's the thing. pyenv just downloads the source code of the specified Python version, and then compiles it on your machine. They did that to be agnostic of the OS you're running on.
It is a single `apt-get install` (which probably downloads less MBs than our typical `npm/yarn` install).
Consider that if pyenv came pre-packaged for ubuntu/debian, those packages would be runtime dependencies and then you would just need to `apt-get install pyenv`.
But having c compiler toolchain available is pretty much standard practice when using python (or nodejs, ruby, etc) because you might need to install some library that requires compilation from source (e.g. psycopg2). If you don't want that, you'll stuck with installing those 3rd party libs from your distro's repo, which might be out of date or outright missing (especially for less popular packages).
Real scenario I'm in right now:
- Need to build libraries to support 2.7, 3.6, plus future proofing for >=3.7
- CI workers have images which only python 2.7 and 3.6, platform team too busy to update them
- Mac Homebrew only supports >=3.7
There's no common environment between anything unless we use pyenv
I understand that virtualenv and maybe even pyenv are useful if you need different requirements for different projects using the same version of Python, as apparently pip installs packages globally. But for your setup, I don't get why something like pyenv really helps...
Yep, most people can just do this. Then, they want a little script over the top that downloads the different versions of Python for them -- just to make life easier for them. Wouldn't it be handy to also script the installation? It'd also probably be useful to automatically setup the version of Python I want to use when I switch folders, so I'm not constantly running the wrong version when I change projects.
... et voila, we've reinvented pyenv :).
It's a Python stack that solves some problems for a part of the community.
It’s a failure of the author to explain intent. The entire article is WHAT, not WHY.
If you are already hyper-familiar with python, you know what you are looking at. Yes, of course I’ll want to use pyenv, “everyone knows that”. For the rest of us, it just seems like a lot of steps to do - for some reason.
I think it’s different from just not being the target audience. A good tutorial explains intent.