Pyenv – lets you easily switch between multiple versions of Python
github.com
github.com
The number of possible modes of failure in this situation is huge.
See also: "Why not tell people to "simply" use pyenv, poetry or anaconda"
https://www.bitecode.dev/p/why-not-tell-people-to-simply-use
I'm not saying pyenv is not a useful tool, but it is not a tool for beginners fighting with python packaging problems. It's a specialist tool to normalize your setup.
Very often, I see people tell me they don't have problems with pyenv, but later on have other unrelated problems with their dependencies. Analysis then prove it was because of pyenv, they just didn't know it. The cost is not obvious.
What is the alternative? Every solution that I know of on Linux requires you to build Python on the machine: asdf, official Python downloads, etc.
I'll leave further discussion of the reliability and provenance of said binaries to someone else.
Still, it's part of those new generation of tooling that bringing hope for the next few years.
Using per-distro official installation channels.
E.g. using deadsnakes PPA on Ubuntu, AUR on Arch Linux, etc.
Those are what Rye and hatch use.
Drawbacks: late availability of patch versions, various quirks from how they are built (missing readline, missing some build info that self-compiled C python modules might need.)
Not only does the format need to exist but the service of building and publishing them is needed too.
For linux, official repos are ideal. If you really, really can't (which is different than wanting to), ubuntu deadsnake and red hat epl are the best second plan, while already more finicky.
If you use something more exotic, you chose hardship, and you will have to be up to the task.
Anything else will come with bigger caveats than the people promoting them will really admit to.
The story is of course a bit richer because linux packaging is fun, so I'll complete with:
https://www.bitecode.dev/p/installing-python-the-bare-minimu...
Thete are currently no silver bullet to bootstrap python on linux. I tried all of them with hundred of colleagues and trainees.
I do have hope for astral to come up with one but don't run on rye thinking your life is saved.
Macports and homebrew Pythons are dependencies of other packages. They can be used by you, but they are not meant for you.
This means at some point, they will contain a surprise, and not a good one.
Sample of one, but I never encountered anything even broken in MacPorts. They seem to embrace the BSD ethos of doing everything right (even if at a slower pace, as some packages lag a few releases behind Homebrew).
Unless you need a Python that's not supported by your Linux distribution, you can just use what's available.
On macOS, MacPorts provides compiled versions for 3.2 all the way to 3.13, as well as 2.6 and 2.7. Right now, I have 3.8, 3.9, 3.10, 3.11, 3.12, and a 3.13 development build. The fact it's not Linux (or x86) might cause some frustration.
My understanding of their comment is that they’re talking about getting older Python interpreters to run on more modern OSes, modern enough that they don’t carry the older Python as a system package anymore. Hence, deadsnakes.
That's indeed extremely specific. I'd imagine this pain could be self-inflicted with C-based extensions that were compiled (and can't be recompiled) with structures that don't exist in other versions.
I don't want to imagine what other eldritch horrors await developers working on this application.
I also imagine the scenario you mentioned causing a lock to a patch version. But I see this as just a normal steady state a lot of orgs drift to if not staying on top of version updates
Drop by and send thanks cause he really needs them.
The big advantage for me is that I can match whatever runtime and standard library a target has (and yes, that's needed more times than not, even in this new age of Docker).
Additionally, you can build an _optimized_ Python. I have this set for my builds:
env PYTHON_CONFIGURE_OPTS='--enable-optimizations --with-lto' PYTHON_CFLAGS='-march=native -mtune=native' pyenv install 3.12.2
E.G, I'm on Ubuntu 20.04 on an Dell XPS, a fairly standard machine, I'll get:
pyenv install 3.9 -v
/tmp/python-build.20240325124651.73089 ~
Downloading Python-3.9.19.tar.xz...
-> https://www.python.org/ftp/python/3.9.19/Python-3.9.19.tar.xz
...
LD_LIBRARY_PATH=/tmp/python-build.20240325124651.73089/Python-3.9.19 CC='gcc -pthread' LDSHARED='gcc -pthread -shared -L/home/user/.pyenv/versions/3.9.19/lib -Wl,-rpath,/home/user/.pyenv/versions/3.9.19/lib -L/home/user/.pyenv/versions/3.9.19/lib -Wl,-rpath,/home/user/.pyenv/versions/3.9.19/lib ' OPT='-DNDEBUG -g -fwrapv -O3 -Wall' _TCLTK_INCLUDES='' _TCLTK_LIBS='' ./python -E ./setup.py build
running build
running build_ext
... etc
But by default it downloads the source, and compiles it.
On top of that it will use a shim which comes with its own world of possible pain.
Again, I don't want to bash on pyenv.
But I do want to lower people's expectations.
It's a tool for experts, not beginners.
Really? I regularly use this on Linux exactly because it compiles from source rather than using the system provided Python which often have patches in them that breaks pip's test suite.
I've never encountered it trying to use the system provided Python, but maybe there's a weird quirk in the way I am using it.
$ PYTHON_CONFIGURE_OPTS='--disable-ipv6 --enable-optimizations --with-lto' PYTHON_CFLAGS='-march=native -mtune=native' pyenv install 3.12.2 python-build: use openssl@3 from homebrew python-build: use readline from homebrew Downloading Python-3.12.2.tar.xz... -> https://www.python.org/ftp/python/3.12.2/Python-3.12.2.tar.x... Installing Python-3.12.2... python-build: use readline from homebrew python-build: use ncurses from homebrew python-build: use zlib from xcode sdk
BUILD FAILED (OS X 14.4 using python-build 20180424)
Inspect or clean up the working tree at /var/folders/w9/xvxzj68j6kx7m480rnwq6hvh0000gn/T/python-build.20240325114837.43577 Results logged to /var/folders/w9/xvxzj68j6kx7m480rnwq6hvh0000gn/T/python-build.20240325114837.43577.log
Last 10 log lines: ./Include/internal/pycore_interp.h:193:24: error: field has incomplete type 'struct _dtoa_state' struct _dtoa_state dtoa; ^ ./Include/internal/pycore_interp.h:193:12: note: forward declaration of 'struct _dtoa_state' struct _dtoa_state dtoa; ^ 1 error generated. make[2]: ** [Objects/boolobject.o] Error 1 make[1]: ** [profile-gen-stamp] Error 2 make: ** [profile-run-stamp] Error 2
PYTHON_CONFIGURE_OPTS='--disable-ipv6
I’m sure he is trying to promote some sort of work flow in that article but I don’t understand which.
> https://www.bitecode.dev/p/why-not-tell-people-to-simply-use
Just curious: what are the downsides of poetry installed with pipx? The article mentions having to install poetry in another venv, but that's hardly an issue with pipx (you just add an 'x' after 'pip'), and installing pipx is as straight-forward as it can be.
Coming from the Homebrew/Ruby ecosystem - Hey @mikemcquaid - installing a entirely separate package manager just to deal with a few projects felt like the wrong thing to do.
Occasionally, I have still needed to compile Python myself in order to get things to work, which isn't guaranteed not to blow up w/ `brew`, but this has become far less common of late.
That is, the pattern seems to be that someone mentions a solution, then a zillion responses as to why it sucks.
Is there any tool or pair that sucks least for most cases and beginners? I get that every case is different, but perhaps there are some useful starting points?
I think his point was that you shouldn't pretend to users that just switching to pyenv is the solution.
It's clearly not because most people successfully use it fine.
The problem of distribution and packaging is often a matter of user expectations vs. the actual problem.
The user expectations is that Python is a high level language and will run the same across different machines regardless of OS and Hardware.
The actual problem is Python is a glue language often depending on lots of libraries that are sensitive to how they were compiled and what hardware they are targeting. So people can end up moving the the glue of the project first and then expect everything else to just work.
Things are getting better (e.g. https://lukeplant.me.uk/blog/posts/python-packaging-must-be-...), a lot of people have put a lot of work into standards and improving the ecosystem, but it is a hard problem that most other popular languages don't interface with nearly as much.
As someone who has managed Python distributions in a large company and who triages issues on the pip github issue page that my anecdotal experience is things are getting better.
The only hard statistic I can point to is the number of top packages on PyPI that offer wheels has substantially gone up, and is close to 100% in the top 500.
And yeah, in general there are significantly less NodeJS third party packages interfacing with packages that directly depend on OSes and hardware. Python has many third party packages that are older than NodeJS that depend on foreign function interfaces, win32 com apis, directly talking to graphics shaders, etc.
Well, no because it's perfectly possible to successfully use a horribly broken system. I use Python "successfully", it just meant I have spend probably literal weeks of my life fighting pip and virtualenv and relative imports and finding I need flags like `--config-settings editable_mode=compat`.
> The problem of distribution and packaging is often a matter of user expectations vs. the actual problem.
Ha yes, I expect it to work reliably and simply and it doesn't!
> The actual problem is Python is a glue language often depending on lots of libraries that are sensitive to how they were compiled and what hardware they are targeting.
That's completely irrelevant to the kind of problems I was talking about. I run into issues with Python failing to compile C libraries relatively rarely! Even compiling Python itself seems to work quite well (maybe not surprising since that's one of the only ways to get a new version on Linux).
It's all the packaging infrastructure that's a mess. Pip, virtualenv, setuptools, and also the module import system is a total disaster. You don't see questions like this for Go:
https://stackoverflow.com/questions/14132789/relative-import...
> https://stackoverflow.com/questions/14132789/relative-import...
I think you'll find the further you delve into it the less the problems are distinct, a lot of the issues with the module import system, pip, virtualenv, setuptools, etc. is because they are designed with having to support a vast range of things, from being depended on to interact with system libraries to downloading sdists and compiling arbitrary languages, etc.
Though the specific example you linked was largely solved with Python 3, there was a lot of confusion during the 2 to 3 transition because people had to support both behaviors, but most people don't have to think about Python 2 any more.
I can assure you it absolutely was not.
> I think you'll find the further you delve into it the less the problems are distinct, a lot of the issues with the module import system, pip, virtualenv, setuptools, etc. is because they are designed with having to support a vast range of things, from being depended on to interact with system libraries to downloading sdists and compiling arbitrary languages, etc.
Not really. There are plenty of systems that have to "support a vast range of things" that aren't this bad.
In my opinion it's because the core Python devs didn't particularly care about the issue, never really tried to solve it, and as a result we have 10 incompatible half-baked third party solutions.
It's similar to the situation with C/C++ - worse in some ways, better in others (at least there is a de facto package registry in Python). In some ways it's because both languages are very old and predate the idea that packaging should be easy and reliable. That's fine, but please don't pretend that it is easy and reliable now.
The question, as posted, was a confusion about how Python 2 relative importing worked, which was indeed bad. I don't know what you think you are pointing out, you haven't said, and the question *is* about Python 2.
> Not really. There are plenty of systems that have to "support a vast range of things" that aren't this bad. > > In my opinion it's because the core Python devs didn't particularly care about the issue, never really tried to solve it, and as a result we have 10 incompatible half-baked third party solutions.
I agree that a lot of the solutions were created when there was not an understanding, or thought out design, on what would be a good packaging solution.
But these ill thought out solutions were exactly because of trying to support this wide range of situations, from working on weird OSes, to integrating with strange build systems.
However, what you seemed to have missed is there is now a first party standard on:
* How package installers (pip, poetry, PDM, etc.) should interact with package builders (setuptools, hatchling, etc.)
* How and where build configuration and project metadata should be stored (pyproject.toml)
Almost every popular Python package tool now supports these standards, meaning they all interact with each other pretty well.
Dropping all legacy configuration, and updating the standards to support edge cases is still a long road, but it is a road being travelled and things are getting better.
The common interface is they all use `docker-compose up` and have their editors hooked into the containers.
And of course you'll have to make sure people equally have good practices, since incorrectly using sudo pip install in docker and not a venv are very common.
So again, one possible solution for a certain context, but I wouldn't sell that to most people. Certainly not to beginners.
People writing their pythonanywhere website won't pop up a container, won't they ?
And btw, JS doesn't have this problem, they use NPM.
Its absolutely, by far, the highest friction alternative on Windows for anyone that doesn't use Docker for other purposes.
And, no, docker isn't just install and run on Windows. Before Docker made a heavy push for paid Docker Desktop for even personal use, it was close to that.
But now the way to get a free usable docker command line is to install WSL and a Linux environment, install docker there, and then invoke docker via wsl. (Which, of course, you will not find via Docker’s own information, which will try to sell you a paid subscription.)
1. Download and install Docker desktop for Windows
2. Restart Windows (guess you don't have to do this with just Python)
3. Run Docker Desktop
4. Say "no" to signing into a Docker account
5. Wait for engine to start, which took a few minutes the first time, a bit annoying
6. Pull and run an image (I tried nginx)
It was weird being asked to log in, but the "no" button was pretty clear. I didn't feel like I was forced to use WSL to avoid paying; maybe they've backtracked from something. Unity was far more convincing that I had to pay.
- Not if you have to deploy the code.
- Not if you have to import the code in your own project.
Basically, not if you want to do anything by running the code. For which I would suggest an installer instead.
It does imply, on linux, to limit yourself to the choices of Python you can install. This constraint is, for most people, preferable than the alternative, even if it gets frustrating to our geeky soul.
For your own projects built from scratch (rather than big multi-dep projects off GitHub), there's a smaller learning curve if you go to the vanilla https://www.python.org/downloads/ , install the latest, and use the included pip to install packages. That'll probably get you very far. It's not like the older days when you needed both Py3 and Py2.
For experts working frequently in Python, tools like Pyenv can make more sense.
But, to add to the list of problems, these Python versions IIRC do not compile with optimisation turned on, so they’re by default quite a bit slower than they need to be.
When the God Of Programming made Python, all other languages were jealous of its elegance, simplicity and intuitive beauty. When the other programming languages went to complain he said..."Wait until you see what package systems I will give them...They will never get an environment properly setup..." :-)
”The CUDA version on your system does not match the CUDA version the realm of mortal men was compiled with”
You will know when you care. These days, doing common tasks, the constraint solver in poetry will often tell you:
“Hey! I can’t find a version of sentencepiece with metadata that lets me combine this with python 3.12.2. Sorry! I give up!”
Now, if you aren’t concerned with using your bare metal for CUDA or fancy new MPS or AMD stuff. Just ignore this and use containers. I’d use podman compose.
However, I use pyenv on every machine. Because it compiles specific versions I actually need, even to create various virtual environments. If compiling python automatically sounds tough, you probably don’t need to anyway.
To describe the problem you’d see. I try to use poetry by default, though I think it became popular before it was PEP-compliant or useful in devops. It is impossible to control the behavior of other package managers, and poetry is/was strict about that. Which means you can’t force deploy in many cases. (Better lately.)
For the problem pyenv helps to solve, I back-up my pyproject.toml setuptools backend with pip and requirements.txt. These days, requirements-cuda.txt and requirements-mps.txt.
The landscape is still a disaster for binary compatibility, but it can be done lol. (I’ve been doing python packaging professionally since prom, which was python 2.6 give or take.)
> I can’t find a version of sentencepiece with metadata that lets me combine this with python 3.12.2
I'm a reasonably advanced python user. I've shipped web apps, Desktop GUIs, cli-tools, and even written cpython extension modules for custom hardware control. I typically target the system `python3` of whatever linux distribution I'm shipping to and I use the system 'python3-virtualenv' for venvs.
But I have never encountered a dependency resolution issue or been forced to use poetry. What am I doing wrong?
It’s always the inclusion of a specific dependency we added for a feature, and based on that dev’s knowledge and experience. It’s often me, but not always.
This doesn’t happen in ecosystems with a base package versioning. This is arguably why anaconda became popular, and why we target base docker images of ubuntu.
Doesn’t work in complex deployments based on money rather than ideals, every time. At least in my career.
Edit: first time I dealt with this, we ended up forking a dependency chain instead of using pip. I lost that war and they ended up reviving a legacy PHP app instead of funding python dev.
For example, I want my own Python apps to work equally well on Debian oldstable (which currently provides Python 3.9) and Arch Linux (currently on 3.11). That means that I’m going to choose the lowest common version (3.9) as the language level for my app or library.
And if I program against a 3.9 language level, I absolutely *refuse* to use any other interpreter than a 3.9 one at development time. (If I used a newer one, then my linter and type checker would helpfully give me false alarms all the time, unaware of the actual language level.)
Hence, I use pyenv and poetry to get exactly the interpreter for the language level I want, and to allow pylint and mypy to be perfectly aware of the language level I’m using at development time, allowing them to give me findings that actually make sense.
Not using python on a system without the necessary build tools while trying to use dependencies with native code that don't have binaries built for your combination of python version and platform?
This used to be a huge problem with Python on Windows, as things that weren't pure Python would very often not have binary packages or have them for a narrow set of python versions, often not the same set as other dependencies. (Not just a windows problem, but it was definitely big on windows.)
Tooling and practices have advanced so that more packages are automatically built against a wider set of targets, so the problem is a lot smaller than it used to be.
Also, the issues the article you link are things that damn near every programmer will eventually need to know. Things like PATH are IMO the basics -- if you don't understand this or how to `$packageManager install` something, you're gonna have a rough time programming in general, regardless of language.
I guess, I'm in the "expert" category. I'm saying it so that people won't be afraid to try it.
> It doesn’t support Windows. That’s game over right there, for half of the community.
What do you mean by this? I use pyenv on windows all the time.
Which ones did I miss? Which of them actually ensure your program always works the same as when you first wrote it, without asterisks?
We still need a generated lock file with every top level dependency and sub-dependencies locked down to their most precise version commit to version control so that when you build your image today or in 6 months you end up with the same result.
Using pip to freeze your dependencies and writing a tiny shell script to generate a lock file at build time is better than nothing to solve this problem with nothing more than pip. It's what I do in https://github.com/nickjj/docker-flask-example and https://github.com/nickjj/docker-django-example. It's not perfect but it solves 80% with minimal complexity.
Of course this doesn't come free: packaging with Nix may involve nontrivial effort for some projects.
nixpkgs maintainers are frequently the first to notify a project author of incompatibility with new versions of another package.
In Nix lingo, a package successfully building or running against an implicit or unmanaged dependency is called 'impurity'. To help keep builds 'pure', Nix builds everything in a sandbox and does various other tricks to ensure that a package being built can't/doesn't find any dependencies at build time that you don't explicitly tell Nix to include in the build environment.
(This is also increasingly how Linux distros build their packages, to solve the same problem.)
For the most part, if building (and running the test suite during the build) a Python package with Nix succeeds, you can be confident that you've got all the dependencies sorted out— even the ones upstream forgot to tell you about.
> Do you have to find them out and fix them manually or can you generate a lock file and hope for the best?
At install time, you can be confident that Nix will bring along all of the system-level dependencies your Python package needs. You don't have to find and fill any gaps at install time 10 years from now or whatever.
When you're writing your Nix package for the first time, you'll be doing a mix of generating the Nix code that defines your package from some upstream, Python-specific lockfile and making manual corrections when the build fails. Nix doesn't have any magic for figuring out dependencies that are left out of poetry.lock or whatever, or for disambiguating guaranteed-compatible exact versions from requirements.txt.
python -m venv myenv
. myenv/bin/activate
<install reqs>
<your program>
And that's the whole problem. "The same Python environment" doesn't mean much because it underspecifies the full runtime dependency chain.
No other solution besides fixed OS (containers, nix) could solve this problem.
You don't have to use NixOS to use Nix. You can install it on any Linux distro, or MacOS, and use it as your build system.
But not the same Python runtime, which does not fulfil the request of guaranteeing it works the same later without asterisks.
Just don't uninstall your python binaries and you're fine.
If you want to package everything up into am image or a zip, you can do that too if you want, but in my 10+ year career I've not had much of an issue just using boring venvs.
It seems that there is a way to do this in pip but I don’t think it is widely used: https://stackoverflow.com/questions/19559247/requirements-tx...
It does not.
> There are symlinks in the environments bin directory to the specific runtime.
Precisely. They symlink to a path. Which means that if you have a certain version of Python at one location (let’s say /usr/bin/python3) and later update that (let’s say by upgrading macOS and the Xcode developer tools), the same virtual environment will point to a different version of Python.
> Just don't uninstall your python binaries and you're fine.
That’s not a practical solution. Sometimes you don’t have a choice, as demonstrated above. By that logic one could say “don’t change anything about your system and you don’t even need virtual environments”. Which is somewhat true, but also profoundly unhelpful.
The point of the question was to reproduce the same thing without asterisks and you’ve introduced a major one and called it a day.
> Error: This build of python cannot create venvs without using symlinks
— Then don’t raise your arm.
Like I said in the original comment you replied to and are now ignoring:
> That’s not a practical solution. Sometimes you don’t have a choice, as demonstrated above. By that logic one could say “don’t change anything about your system and you don’t even need virtual environments”. Which is somewhat true, but also profoundly unhelpful.
- Doctor, it hurts when I hit my head on the wall
- Then don't hit your head on the wall
You have solutions available, use them
— Hey, so you know how we have this requirement, which came about after years of dealing with and understanding a problem and the available solutions?
— Yes, what about it?
— Well, a random commenter on Hacker News who has zero context of the problem has suggested we use a method we already found inadequate for our specific use case.
— Oh wow, in that case let’s replace the whole system right now.
I'm not ignoring you; I just don't think your use case of "I want to capture the Python from Xcode in my dev env so it's resilient to changes and upgrades" is something anyone wants (or should want) to do. Do you really want to ship on that version? Are you asking your users to install Xcode?
> That’s not a practical solution. Sometimes you don’t have a choice, as demonstrated above. By that logic one could say “don’t change anything about your system and you don’t even need virtual environments”. Which is somewhat true, but also profoundly unhelpful.
In other words: when are you forced to use Xcode's Python for development, and why would that be a good idea? I'm earnestly asking; there may be reasons; I'm just not aware of any.
Then you are wrong. Simple as that. I’m describing a very real scenario.
> Do you really want to ship on that version?
Holy moly, is it really that hard to understand the difference between wanting and having to? For the use case, and older version in a consistent place which is easy to install is the best solution.
> Are you asking your users to install Xcode?
Triggering an Xcode CLI tools installation is simple and done graphically. And it’s one step removed from installing Homebrew or pyenv, which both need them (even for the scripted installation, pyenv requires git).
> In other words: when are you forced to use Xcode's Python for development, and why would that be a good idea?
See, for a moment there you understood it’s about users, not just your own dev environment, but then went back. Unfortunately, after this conversation with you two I no longer have the energy to go through it in detail in an uphill explanation. Another time, maybe.
> Then you are wrong. Simple as that. I’m describing a very real scenario.
>> Do you really want to ship on that version?
> Holy moly, is it really that hard to understand the difference between wanting and having to? For the use case, and older version in a consistent place which is easy to install is the best solution.
Tone is hard on the internet, but I'm honestly trying to understand your use case. The way I understand it now is "I want to be able to develop Python programs locally using only Xcode's Python." I'm still not sure why you want to (generally when people ship Python programs they bundle a runtime) but let's set that to the side. For this use case, I don't understand why symlinks don't work for you. Xcode installs to versioned folders so you can have multiple Xcode installs side by side, that way new versions won't overwrite things. I'm obviously not an expert here though; am I missing something?
I never said that. My assertion was that venv does not ensure the same Python runtime.¹ That’s it. I don’t have a problem with that. But I do know of one situation where it could make a difference and “do it another way” is not a reasonable answer.
If we ever meet in person I’ll gladly explain it in detail.
It's python3.11, not python3, and the python3 "executable" is, itself a symlink to the particular binary (in this example python3.11). Upgrading just changes the symlink, which wouldn't affect the venv, which isn't using python3, it's using python3.11.
I didn't introduce anything, I explained how the links are to the versioned binaries, which isn't what you're stating is happening.
Edit to add: as someone else also points out, you don't even have to use the symlinked versions, you can use --copies
Not for the example I gave. If you’re seeing Python 3.11, you’re definitely not using the /usr/bin/python3 on macOS with is made available by the Xcode CLI tools. That’s at 3.9.6 even on Sonoma.
> as someone else also points out, you don't even have to use the symlinked versions, you can use --copies
Which, again, doesn’t work for the stated case.
No, it is you who are failing to understand. I’m describing to you a real scenario, but it’s not one that bothers me. I don’t need you to come up with a solution and didn’t ask you for it. You need to understand not everyone has the same requirements and tradeoffs you do.
You're supposed to install a specific version of python in a specific place, with a specific name. Say, /usr/local/python-3.10.6
Use pyenv to use that python. Control that by creating a `.python-version` file that says 3.10.6
You now have a project that uses 3.10.6. Unless, of course, somebody installs a different version in that path - at which point you've got bigger issues
Using pyenv to use `/usr/bin/python3` and hoping for the best misses the point
Because that’s not what the conversation is about. It’s about virtual environments.
Forget to install virtualenv? Have fun getting rid of packages in your global system.
Sub-dependencies? You'll have to think of a strategy for it. Most people don't and the reproducibility of their project suffers.
Developer vs production dependencies ? You'll have to think of a strategy for it.
And I think I'm forgetting one, but that's already enough.
Using python and python libraries only from your package manager (like APT) for a specific OS version.
Micromamba as well, lightweight version of conda/miniconda.
Only pdm and poetry generate cross-platform lock files by default as far as I know, but there are a lot of people trying to solve this problem right now.
It's not an easy problem to solve. Python's package management predates package managers from most other programming languages and Python itself predates Linux. There is a lot of baggage so change is very slow.
I think rye hits most of the points pretty well, it ensures both python and package versions.
Nix does, if you want to actually invest the effort into the "without asterisks" part.
Edit: Wow y'all are some sour people for voting down this question. I truly wonder how often people run into this problem compared to just complaining about it.
In my edperience I had many many problems with OS packages vs pip installed ones. There were really strange dependency issues.
In some cases I encountered dependency hell.
Even if somebody said to me that the core means also using virtualized en I would disagree as places over Internet not always explicitly guide you in that route.
NB. The latest incident was Friday when I've discovered that some CI pipeline ran `setup.py install` that down the lane invoked easy_install, which doesn't have a policy of ignoring bizarre versions s.a. X.Y.Zrc1 or X.Y.Zb2 etc. It ran aground when it was trying to install scikit-learn which wanted NumPy >=X.Y.Z, but it already installed X.Y.Zb1, and it didn't realize that this version should be OK (also, it shouldn't have installed non-release versions anyways).
I run into some kind of packaging or dependency problem for basically every non-trivial project. Hence the many "solutions", but somehow most end up worse than the problems they attempt to solve
Standards ensure that going forward language semantics and syntax don't change. Having minimal dependencies ensures program longevity. Package manager cannot solve these problems, no matter how good it is at its job.
Python doesn't have a standard, it's heavily reliant on dependencies which are very plentiful and similarly unregulated. A program written in C that uses only functionality described in some POSIX standard will endure decades unmodified. Even Python helloworld program went stale sometime ago, even though it's just one line.
Or the "says it will work with > X," but doesn't.
This is, of course, an infrastructure/maintenance issue as much as a package manager design issue. But in Nix's case, the public 'binary cache' (for Nixpkgs/NixOS) of build outputs includes not only final build outputs but also the source tarballs that go into them. As Nix disallows network access at build time, all dependencies are represented this way, including jar files or source tarballs, or whatever— Nix itself must be the one to fetch your dependencies. Consequently, everything you fetch from the Internet for your build is a kind of intermediary Nix build that can be cached using the usual Nix tools. The Nix community's public cache has a policy of retaining copies of upstream sources forever (there is recently talk of limiting storage of the final built packages to a retention period of only 2 years, but sources will continue to be retained indefinitely. So far the cache reaches back to its inception around a decade ago.)
Taken together, these things mean that when a project disappears entirely from GitHub or Maven Central or whatever, people building against old versions of it with Nix/Nixpkgs don't even notice. Nix just fetches those upstream sources from the public cache without even reaching out to that central repository from which those sources have been removed.
For private use cases where your project and its dependencies won't be mirrored to the public cache of Nixpkgs builds, you can achieve the same effect by running your own cache or paying a hosted service to do that.
For builds outside the Nix universe, you can make special arrangements for each type of package your various builds fetch, and mirroring those repos. Then configure your builds to pull from your mirrors instead of the main/public ones.
For me, the combination of asdf and Poetry has worked quite well recently: I use asdf to pin the Python & Poetry version and then use Poetry to pin everything else.
https://github.com/asdf-community/asdf-python?tab=readme-ov-...
It’s no panacea, but feels more stable and usable (especially from onboarding new team members PoV) than other tooling I’ve tried
The generated virtual env name is a little wonky. I'm not sure exactly what the scheme is, but it's basically `$(package)-$(some sort of hash?)-$(pyversion)`. I can't speak for all tools, but at least VS Code detects the poetry env and suggests it (and indicates it as a Poetry env) when you go to configure the project interpreter.
There aren't such tools. Python not being a standard and heavily reliant on the OS that runs it and on third-party components that are also not standard leaves you with no choice by to "be at the wheel" all the time. Virtually anything written in Python will go stale in a mater of few years. In other words, you need to constantly update and test as the environment changes just to stand still.
The other one is Docker, which some people hate. I don't mind it, in some cases I prefer it.
requirements.txt is a text file, not a separate tool...
if you want your program to work exactly as intended in the future, there are tools like py2exe and py2app for that.
On my personal machine I don't bother with venvs, so also haven't thought about it in years. But was trying to figure out what the "workon" command did for GP.
this way I have all my envs in ~/virtual and all my projects in ~/projects and just `workon xyz` when I want to be in a certain venv for a given project (which doesn't always map one-to-one)
Edit: Oh I see you didn't. I can't read.
Needing more than one, MAYBE two versions of your language installed is insanity. If I build a Python widget and send it to my non-programmer coworker, there is exactly zero chance it will work until I walk over and manually set up the correct language. Instead I use a language that natively compiles to an exe.
Python is pretty neat, but the concept of a program that just works anywhere is so utterly alien that I genuinely cannot find any practical use for it.
I can't just install a pip package to use globally, I have to set up some goddamn virtual environment because everything is so brittle that nothing works without the exact correct language version.
It's like NPM all over again. Dependency hell is so bad that your package manager is now a dependency so we need a package manager manager to manage installing the correct package manager version to then install the correct packages.
Every single time I've come up to some Python thing or other, I spend about fifteen minutes fucking with it before giving up and using a tool built in a sane language.
The fact that I can't just send any random person a program and expect it to run at all is just lunacy. And no, compiled Python does not count because that's even more brittle tooling on top of all the other bullshit I have to deal with. If it doesn't work out of the box, it's a bad tool and I have far better things to do with my very valuable time.
I am only half-kidding. But it appears the industry as a whole is concentrating effort on fewer languages and Python is one of them. If I want to distribute my app - I write it in React, but I still often use python in the process: prototyping, asking GPT to rewrite X from python to typescript, making mock servers for tests / preparing data for unit tests.
(Also I like mise better currently: https://github.com/jdx/mise)
Even if you just use mise as an asdf alternative, it has a nicer CLI and interacts with the same plugin ecosystem more smoothly.
(I've found it to be an improvement on direnv, but I still use make)
the comparison to asdf page has more: https://mise.jdx.dev/dev-tools/comparison-to-asdf.html
If I'm keeping the projects under Dropbox, then I'll just add the .venv folder to the ignore list ("attr -s com.dropbox.ignored -V 1 .venv") to reduce the amount of files that need to be synced. If I need to get back to old project, I simply recreate the venv and install dependecies using Poetry.
A good habit would be to add a .pythonversion file on the project folder. Pyenv can pick that up and it is then obvious which version was used for the project.
I do development on Windows, but run all the Python stuff on WSL2 and use VS code for notebooks and everything. One thing I haven't looked at is using pipx to manage Poetry. Now I simply have one Poetry version installed.
Pyenv has worked fine once I managed to collect all the necessary dependencies to Ubuntu. For me it's not just to switch between Python 3.x. I like the fact that you can easily have different patch versions and match exactly the version I would use in production (via containers).
Our flow is similar: pyenv (windows and linux), pipx, poetry.
We’ve also defaulted poetry to utilize the current global version of Python and build the venv within the project folder.
pyenv exec pip install poetry && pyenv exec poetry install
And every time that `.python-version` changes (which I at most do a few times per year per project,) I throw away the `.venv`, do `pyenv install -s` and start over.
Your `poetry run` point still stands, though.
Python really makes things difficult :D
You’re absolutely right.
On second thought, I guess I only need these Poetry installs just once anyway: for creating the venv and then never again. That’s not worth it.
So here’s a new method that does away with these extra Poetry installs:
pyenv exec python -m venv .venv && poetry install
Going to adopt this variant from now on, I guess. Thank you for your thoughts!
They will try it, and most of them will pay the price for it.
It's a cycle.
https://mise.jdx.dev/lang/python.html
It has other nice helpers for development environments (tasks, env variables, etc.).
- Download whatever Python binaries I need from python.org
- virtualenv --python=$PYTHON_VERSION ~/.virtualenvs/$PROJECT_NAME
- pyproject.toml
- pip-compile --generate-hashes
There has been tons of churn in the Python project management space, and I feel like I've been blissfully unaware of all of it with this workflow. Can't recommend enough.
Why regular Python binaries and not pyenv? I've had enough pyenv snafus and meltdowns that I started looking for alternatives, and it turned out I could just install whatever Python binary I wanted and specify it wherever I wanted. What could be easier?
python -m venv is fine, I've just gotten used to virtualenv.
pyproject.toml is the future and almost everything supports it pretty well; it also has fewer implicit weirdnesses than setup.py or setup.cfg.
pip-tools are really simple, and they let you have hashed dependencies which are super important IMO.
It's also my understanding that pip-tools does not let you make layered requirements files with pyproject.toml files, only with .in files
I guess I don't know what you mean by OS stuff, but maybe this works for that too?
I don't recall ever needing anything other than virtualenvwrapper and pip—and even some of the annoyances these tools had early on have been solved by now...
https://virtualenvwrapper.readthedocs.io/en/latest/
If you really need different versions of python, you can just `mkvirtualenv -p python3 venvname`
I feel like every other tool out there has to explain what problem they solve that virtualenvwrapper doesn't
Don't install anything globally, creates lots of envs, and feel free to have different versions of python installed side-by-side with some "main" version preferably symlinked as `python` and `python3`
The end
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
For example, if you need the latest version of Python and it is not available through your OS:
pyenv install 3.12
pyenv global 3.12 brew install python@3.8
brew install python@3.12
on Linux it should be just as easy to have multiple versions side-by-side
it's been a while since I've done something similar on Windows but it's also possible (although I think I would prefer to use WSL if I were on Windows these days)
This is what pyenv does.
also it's terribly named because it conflates the idea of python environments with python versions
just install pre-compiled python from official sources side-by-side. no pyenv needed
and I already have homebrew to install python (and anything else), which I trust to have more users and better formulas. other OSes have their own tools
This is the same with a binary though. And with homebrew, you can't follow patches or flags used or if they change.
- https://github.com/Homebrew/homebrew-core/blob/c964ad7fa53ad... - https://github.com/Homebrew/homebrew-core/blob/c964ad7fa53ad... - https://github.com/Homebrew/homebrew-core/blob/c964ad7fa53ad...
Not arguing for good or bad, just saying that the problem isn't limited to this.
It also ensures that my requirements.txt is sufficient for the code to run.
The shortest ritual I could figure out so far is:
pyenv install -s && pyenv exec pip install poetry && pyenv exec poetry install
Now I understand I should have been using Pyenv the whole time. But it sure felt unfriendly for a newcomer to the language/ecosystem. Granted my learning approach is to jump in without much preparation, I wish the language designers or the ecosystem provided a better experience when things don't work right.
Pyenv allows you to use whatever version of Python you want.
On most Linux distros you probably don't want to change your system python to whatever, because there are a lot of system programs that may not be compatible with a version other than what is on the system by default.
Then once you activate it you can make sure your python
just install the different python versions by downloading the binaries or through your package manager and run python3.<whatever_version_you_want> -m venv and the virtual environment will automatically, forever, use that python version when you activate it.
Many developers, not just me, have a similar setup: we use virtual environments everywhere, and if you aren't in one, `python` doesn't even resolve to a symbol.
If I want to write a quick script with no dependencies, I directly call `python3.xx` on it. Otherwise, I create a virtualenv.
Yes, it's a bit harder for beginners, but from a huge amount of experience helping people who are starting up in programming, people have little issue in following a few more instructions. What demolishes beginners is getting into a bad state where nothing works and you don't know why.
Virtual environments are for your installed dependencies, whilst pyenv is for installing python.
I have a client that uses Python X and another that strictly uses Python X+1.
The virtual environments are so that I can have the project dependencies installed and the pyenv lets different companies have different cadence for their Python upgrades.
I could be completely mistaken and mixing up my Python support utils as I've not had a client request Python for a couple of years.
Just install several pythons with different binary names.
ex. python3.8, python3.9, python3.11
No need to complicate things.
If using Poetry, you can just do `poetry env use python3.8`.
> $ pyenv install 3.8
> $ pyenv local 3.8
> $ python3 -m venv .venv
> $ source .venv/bin/activate
and you're done. Not really sure what you might consider difficult about pyenv, but it's just a tool to instigate your python venvs which is a built tool for most modern versions of python.
- compiling from source
- need to install build-essentials on docker images which takes up lot of space and takes a long time
The good thing about pyenv is that it makes you rely less on package maintainers. You can come back to your Python project 10 years later and have a real chance that it still works, even if you have changed workstations in between.
The main scripts are `venv-create.sh`, `venv-delete.sh` and `bootstrap.sh`. `venv-reset.sh` pulls these three scripts together to make reinstalling your venv a single command.
Here's the link if anyone is interested: https://github.com/adap/flower/tree/main/dev
I'm not using things like anaconda for a similar reason I'm not using docker. -- I don't want a separate OS install for each service I'm running on my server. It works perfectly fine.
So fast it finally made virtual environments usable for me. But it's not (yet) a full replacement for conda, e.g. it won't install things outside of Python packages
A notable open issue in poetry is we can't currently specify one dependency on torch, and have it grab CPU version on some systems and GPU on others. Does PDM solve that?
[tool.poetry.dependencies]
python = ">=3.10,<3.12.0"
torch = {version = "^2.0.1+cu118", source = "torch118"}
torchvision = {version = "^0.15.2+cu118", source = "torch118"}
[[tool.poetry.source]]
name = "torch118"
url = "https://download.pytorch.org/whl/cu118"
priority = "explicit"
poetry add torch torchvisionHowever, this works perfectly with Poetry 1.8 and Pytorch 2.2. I suppose the only problem is what PDM also does, where the lock file is platform-dependent. I'm not sure whether Poetry allows you to select a specific lock file, however.
This link shows which package versions come in which Docker tag and is invaluable: https://docs.nvidia.com/deeplearning/frameworks/support-matr...
Nowadays, most DS people only want to do ML at the experimental stage only and get lost when things get on the engineering side of things. But for their defense, nowadays the bare minimum skills require to do programming, containerization, CI/CD, etc. More experienced and swiss army knife SWE/MLE have to educate the willing.
It was already the same 10 years ago with MATLAB dudes not wanting to get dirty with C/C++/ASM SIMD. The history repeats itself, only at a faster pace
python -m pip install torch torchvision
There's a very good chance I've installed cuda on my system before this though. And usually cudnn and some other packages because this is part of my standard install. And then I also never run into the issue where a package is looking for nvcc.
We make extensive use of conda/mamba to solve this, and are pretty happy with it, especially with conda-forge.
It's definitely a learning curve, but it turns out every conda user has been bit by the irreproducible tendencies of conda quite often. Nobody uses the conda env file, they just start an env and pip install things into it. They don't realize the base env has stuff, too, and conda envs are hierarchical rather than isolated. I know it's possible to use conda in an isolated and reproducible way, but have yet to meet someone that does so.
So it hasn't been hard to pitch poetry to these folks, and while many complain about the learning curve they appreciate the outcomes.
We're a pytorch shop, and torch mostly just works with pip or poetry these days, as long as you skip the versions the torch maintainers mispackaged. We rarely need anything higher-level that only conda could install.
We really like having more than two dependency groups as this allows us to keep research and production in the same repository. main, dev, research. Then researchers contribute to the core library of a project and keep research and production using the same code for running and evaluating models.
I haven't been using anything CUDA, but the scientific geospatial stack is often a similar mess to install, and it's been handling it really well.
These tools have the advantage of not being multi-taskers and can manage version for all your tools. You wouldn’t need pyenv and npm and rvm and…
We’ve even started committing the .mise.toml files for projects to our repos. That way, since we work on multiple projects that may need multiple versions of the same tool, it’s handled and documented.
All that being said, the creator of Rye is 100% cognizant of that XKCD comic, this [1] is a nice read.
I'm not super well versed in Python tooling at all. I've had to work a lot in Python in the past 6+ months, and I become super confused when I tried making a Python project in my spare time.
I settled on Rye because it just seemed to be the easiest to use.
[0]: https://rye-up.com/ [1]: https://github.com/astral-sh/rye/discussions/6
From my perspective, it doesn't come easier than running a batch file to switch between python versions.
unrelated rant -
Why is the "requirements.txt" file a stupid flat listing of all transitive dependencies with pinned versions? It makes it harder to change library versions even if there are no true conflicts. I've resorted to making an "actual_requirements.txt" file manually listing only direct dependencies and whatever version constraints make sense. I wish pip would fix this.
Pip-tools has an elegant solution for this. You write an requirements.in where you list packages and optionally versions, then pip-compile turns that into a requirements.txt with every package including dependencies specified by versions. You can use it to update specific packages as well.
My friend, here is what you seek: https://github.com/jazzband/pip-tools
requirements.txt is flat because it's really the output of `pip freeze`. It's supposed to completely and exactly rebuild the environment. Unfortunately it's far too flexible and people abuse it by putting in only direct dependencies etc.
If you're writing packages, you don't need a requirements.txt at all, by the way. Package dependencies (only direct dependencies) live in pyproject.toml with the rest of the package config. requirements.txt (and pip tools) are only for when you want to freeze the whole environment, like for a server deployment.
Someone needs to standardize a single Python environment at least annually. As these tools are just turning Python into a perpetual Alpha build.
Best of luck, =)
One of the troubleshooting steps when "something mysterious" happens on developer's computer (eg. package installation fails for inexplicable reasons, Python "standard" library components missing or present when shouldn't be, incorrect component version etc.) was to remove pyenv.
This step was often met with resentment and arguments... but, in most cases I was able to win :)
----
Now, here's a larger point: some programs solve the problem by actually solving the problem, while other programs solve the problem by adding more code around the problem, which, usually, creates new problems while only partially solving the original problem.
An example of the former: fsck -- you run it, it looks at your filesystem, tries to fix it, if it's broken and then gets out of the way entirely. An example of the later: Kubernetes -- you start by having a problem of resource allocation / management and you end up with a problem of resource allocation / management compounded by problems with component version management, configuration management etc.
pyenv falls into the second category of programs. The problem it's trying to solve is: install and use multiple versions of Python. There's really no need for an extra helper program to solve this problem. Multiple versions of CPython can be installed and used together without the use of any extra tools. I.e. the solution to this "problem" is simply to learn how to do that.
Those who advocate for the use of pyenv and the likes usually make an argument for "simplicity". I.e. in their mind, not needing to know how to install multiple versions of Python is a bonus. Something that, potentially, saves them several hours of reading the documentation and perhaps saving them a tiny bit of typing when setting up Python initially.
I contend that this calculation is off because it doesn't account for the problems down the lane. In other words: pyenv helps until it doesn't, and then it becomes a liability. Debugging is always more difficult if you have more wrappers between you and your problem. So, while individual users will not face a lot of problems with "wrapper solutions", those who service such users will face such problems a lot more frequently. That's why, as an ops person, I really dislike "wrapper solutions" -- for me, they complicate the task, never help.
How do you do that? For example, if you're on Ubuntu LTS but want to use the latest Python version, how do you do that? The system package manager won't have it. Do you rely on a third-party PPA? What about other distros, where you don't have that?
But, you can also download CPython source tarballs and skip the Git part. I haven't used CPython installers in a very long time... but, I'd imagine that the MSI would have a way to specify at least the location where Python is to be installed...
Only marginally less surprising than seeing a link directly to python.org.
Then I found Pipenv. Sticking with it. (Emacs has support for this).
Also, this really isn't a problem -- just blast out of the old venv, make a new one with a new version, and you're good.
And then the same for your friend the buildserver.
It's all about reproducibility.
in the "worst" case you can always do a docker
So you actually need something that can read a specification of the required python version and use the correct one from the available options (maybe even reaching out and getting it if it isn't already locally available, though I don5 remeber if any of the existing python build tools will do this; docker obviously will, but its not really a python build tool), across different OS flavors (maybe not that last bit, depending on the use case, especially for internal development), and with minimal overhead for the build tool itself.
allpy --std=py3.4
gcc -std=c90
I mean, imagine if C compilers required that you have a separate compiler binary and copy of the standard headers (or even worse, all of stdlib!) for each version of the C standard?!? Instead of just putting new language features on version flags, and make stdlib functions available/marked as deprecated based on version checks?
For now I'm not seeing a lot of reasons to use PyEnv over the `venv` module that ships with Python 3.3+ [1]. I'm sure there are some thing PyEnv does that `venv` doesn't, but the fact that `venv` ships with Python greatly simplifies things.
Basically my workflow is:
1. I'll create a Python project with, `mkdir dirname`, `cd dirname`, `git init`, `python3 -m venv .env`. This creates a hidden folder named `.env/` which contains the virtual environment. I'll then usually add `.env/` to my `.gitignore`, and also run `pip install --upgrade pip` and `pip install wheel`.
2. If I'm using an existing project, I'll do `git clone `projname`, `cd dirname`, `python3 -m venv .env`, `pip install -m requirements.txt` (requirements.txt is the idiomatic name for the dependencies list in Python projects).
3. I have the following lines in my `~/.bashrc` (hidden file that contains Bash settings):
# Gets a directory named .env or .venv if it exists in the currend directory or any of its parents
get_env() {
if [ -d "$1/.env" ] ; then
echo "$1/.env"
else
if [ -d "$1/.venv" ] ; then
echo "$1/.venv"
else
if [ -d "$1/.." ] ; then
get_env "$1/.."
fi
fi
fi
}
get_absolute_path() {
python3 -c "import os; print(os.path.realpath('$1'))"
}
on_prompt() {
# Load a virtualenv environment if it exists in a file named .env
env_folder=$(get_env $(pwd))
if [ -d "$env_folder" ] ; then
if [[ $VIRTUAL_ENV != $(get_absolute_path $env_folder) ]] ; then
echo "Activating env '$env_folder'"
source "$env_folder/bin/activate"
fi
else
if [ -d "$VIRTUAL_ENV" ] ; then
deactivate
fi
fi
}
# Call on_prompt() every time the command prompt executes
PROMPT_COMMAND=on_prompt
4. When I install a dependency, I use `pip freeze` to get the dependency string, and I append that line to the file `requirements.txt`. For example, if I do `pip install django`, I get `Django==5.0.1` in my output from `pip freeze`, so I'll append that line to my `requirements.txt`.
5. To upgrade dependencies, I edit the version number in the `requirements.txt` file, and then run `pip install -m requirements.txt`. This makes sure that the requirements file stays up-to-date with my locally installed dependencies.
6. To get valid version numbers for a file, you can do `pip install <package>==`, which is basically asking pip to install an invalid version number. This causes it to list valid version numbers. For example, you can do `pip install django==` to view all available Django versions.
I don't get why you just don't set a minimum version like every other manager. Or if you want to do this crazy thing, don't rely on package maintainers to enforce it. I just ends up in major bloat
‘brew pyenv-sync’ may be useful around this area, too.
This is practically the only remaining annoyance I have with the Python ecosystem (relative imports aside). I use some tools, like Glances [0] whose formula relies on a much newer version (3.12) than the actual package requires (3.8) [1].
So when there's a Python update, all of those update as well. I thought I'd fixed this with pipx, but in a way that's worse, because the venvs it builds depend on a specific version of Python existing, which doesn't work well with brew always wanting to upgrade it.
I want a stable, system-level Python that I don't touch, don't add packages to, and which only exists as a dependency for anything that needs it. If an update would break a package I have installed (due to Python library deprecation, etc.), it should warn me before updating. Otherwise, I don't care, as long as any symlinks are taken care of.
Separately, I want a stable, user-level Python that I can do whatever I want to. Nothing updates it automatically. I can accomplish this by compiling Python and using `make altinstall`, but if there's a better way, I'd love to hear about it.
[0]: https://github.com/Homebrew/homebrew-core/blob/20e744191e74d...
Just looking at a random formula, am I correct to understand that this will use python 3.10 and NOT 3.12?[0] I understand ones like this[1] where there's a note about the issue with newer versions.
What I'm trying to understand is if the python version is specified by the formula or it will default to the newest version. If it requires it to be specified in the formula then doesn't this make it contingent on the maintainer upgrading it every python version? I didn't verify [0], but it looks like it should work with later versions of python, and it it is still using 3.10 then isn't that essentially the maintainers "fault?" Because that's my concern. I can't see how something like this stays updated when it requires a maintainer to update. Seems better to have a >=3.10 and then do ==3.10 if only 3.10 works (odd) or >=3.10 <3.12 if it works for 10 and 11 but not 12. Especially since formulas are often made by people that are not the package developers themselves and we're just reliant upon someone keeping up. I'd rather break upon a new version than have many old pythons installed. I know there's no perfect solution, but we're talking about failure modes. And fwiw, I'd rather it try to use the system or environment python rather than installing a unique version. It just gets confusing when you need to add dependencies for optional stuff that wasn't included in the formula and is extremely non-obvious to a new user.
[0] https://github.com/Homebrew/homebrew-core/blob/12a0f6bbbeda8...
[1] https://github.com/Homebrew/homebrew-core/blob/12a0f6bbbeda8...
(I am a Homebrew maintainer).