It feels like it is one of the reasons experienced devs are ditching Python for production systems. Besides horrendous performance and lousy semantics. The cost of setting up, maintaining the environment and onboarding people is just not worth it.
* External state is changing such as project contributions
So its not a one-off unless the project and dev environment is static. The real problem is different tooling doing different amounts of hand holding and automation. Your editor may configure some things automatically, brew may configure some things automatically, and so a set of instructions for setup or problem fixing could be voided without the end user knowing. So now you're off on a adventure of unknown duration wading through internet forums trying to find the resolution that works for you.
Ironically using Docker to isolate the environmental changes is a approach some people use to avoid some of this esoteric crap.
I'd even dare to say that Docker _is_ the answer to the Python's packaging problems, and might have never taken off without that "killer app" that is the Python packaging sh*tshow.
But... I'm not going to hold you in the dark: the reason and the major drive to have a packaging system is that you can define dependencies between packages s.t. users installing package can coordinate and install the stuff they need. Docker simply doesn't do that. You get images. They are completely independent. Whether any two images will work with each other is anyone's guess.
Docker doesn't solve Python's packaging problems by being a better packaging system. It solves it by sidestepping the issue of users installing packages themselves, and shipping the disk of the one machine where it once worked.
It's a bad linker.
(not a fan of the condescending tone in your post, btw)
Lol. It doesn't. How are you going to create Docker images? Run more Dockers inside Docker? And then Dockers all the way down? You still need to install the packages... just while creating an image. Makes absolutely no difference whether you do it inside or outside, you still need something that can install and coordinate packages. Docker is simply irrelevant to this problem.
Once. On a build server. Or a dev's workstation who had the time to fiddle with all the breakage that is python envs. This absolutely does sidestep the issue of distributing Python packages to your users.
We're talking about different problems here. You are talking about library dependencies for devs (and I understand that I'm a bit off topic for this thread)
I didn't ask where, I asked how. How will you know what packages do you need to install? Are you going to go on pypi.org, download them one by one, unzip, look for requirements, then go to pypi.org again, download more, and so on, until you have enough packages and have figured out all the requirements? If so, I have very bad news for you: your single installation might take weeks, possibly months, or might in fact be so slow it will never end as new packages will be published faster than you can identify the version you need.
> This absolutely does sidestep the issue of distributing Python packages to your users.
No it doesn't... OMG. You cannot be serious? Do you really search and solve dependency problems w/o using any dedicated software? Like in your head? Or with pen and paper? For any non-trivial project the number of combinations you have to try before you find the one that works will be astronomical...
> We're talking about different problems here.
Nah. You just don't understand the problem. It's similar, but not the same.
I’ve always seen it recommended to use pyenv or just download directly from Python.org instead.
Python barely survived becoming the default scripting language on Red Hat and other Linux distros which was a major obstruction to the Python 3 transition. If the new cohort of pandas and scikit-learn users had not been such a force of nature we”d be talking today about Python the way we do about Perl.
Not installing venv is a serious crime on the part of Debian as beginners don’t need any more excuse to trash their system with site-local directories and other wrong answers for how to manage packages in Python.
If somebody wanted to package a modern Python application as Debian package they’d have to make a Debian package that contains several Python packages maybe as a venv or executable wheel, it is a solvable problem but a bit like comparing tic tac toe to 12d chess with supersymmetry in terms of the ambition of Linux distros if not the difficulty.
If you installed python packages as debs without any isolation you'd never be able to install two applications that depended on different versions of the same Python package.
The best thing about Java is that it is so xenophobic that nobody expects to have a working Java runtime installed with the OS so you never have to deal with a broken runtime installed with the OS. JAR files are remarkably self-sufficient so "write once one anywhere" is way closer to the truth than it is on "DLL Hell" platforms like Windows and Linux.
pip install —-user
The only time I've ever had a big mess with broken Python installs was after using brew on Linux - luckily killing off brew brought the system Python back fine. I'll grudgingly put up with brew on a Mac out of necessity, but keep it away from Linux.
That said, some Python packages rely on native code that you might find yourself brew-installing. That can be a nightmare.
My typical count is 2-3 per project per machine I work on (could be anywhere from one to one hundred). Then there's a different number of people who need to set up these environments on different machines too (and sometimes require my support).
So, the answer is: who knows?
> putting everything in a docker container
I think, you meant image, not container, but that's an easy mistake to make. And the answer is: both options are terrible. It's a choose your poison kind of thing.
> cool kids
My impression from working with the cool kids is that our industry selects for particular taste in t-shirts rather than knowledge or experience. I'm afraid that the correlation might swing into the negative, if we take either knowledge or experience vs coolness.
---
Most importantly: venv is not the problem. It's a bad fix for a problem. Bad as in it doesn't fix the problem actually, it pretends to. I mean, maybe it covers some 90% of the problem -- who knows, I didn't count. So, it kinda works for a lot of people. But, honestly, I'd prefer that the problem was fixed s.t. it doesn't require venv. It's kind of like discussing various dental prosthetics options: it's better to just have healthy teeth.
I have yet to come across a situation where I need a virtual environment at all. A lot of projects use it, but then lazy me just runs git clone && python3 clone/main.py and it works just fine, sometimes after an apt install python3-somedependency or two.
It always seemed weird to me to depend on such a specific version ("package foo can't be older than 7 months or newer than 4 months"), how even does one manage to use such obscure features that they were removed and so you need an older-than version?
And then there's the people in this thread (seemingly a majority when judging by top level comment loudness) that have trouble with virtual environments and so add another layer on top to manage that virtual environment thing. The heck. Please explain
I haven't tried out virtual environments yet.
I do remember python2-only being a problem back in the day, but this was solved... hmm, maybe in 2017 somewhen? At least for the packages I used then that had py3 issues before, like sklearn, numpy, and scapy come to mind. I think it more or less coincided with Debian deciding the next release was not going to ship Python 2. Somehow that made everyone 2to3, fix a few remaining bugs, release, done. I'm too young (I'm 30) to really have done much with Python 2 so I didn't have this legacy in my own code (besides a few early files), I've ~always just tried to find and install Python 3 versions of everything.
Maybe everyone has come to think we need layers on layers on layers because management tools (like venv) are blogged and talked about, whereas it's a bit dull to write/talk about nothing (i.e. not using/needing such tools)? I genuinely wonder