This seems less like a reason to use `python -m pip` and more a reason to endorse languages with saner versioning and less need to sandbox twenty different versions from one another.
That being said I would like to inquire which languages _you_ could give as an example that work perfectly well with the system-provided compilers when you are using the latest features.
Honestly virtualenv's solve most of these problems. If using a virtualenv seem to low level or a pain, I can't recommend Poetry [0] enough. It uses virtualenv's in the background, and has from my experience a great dependency resolver.
All of those build tools are pretty complicated. I honestly prefer python and virtual environment. Simple requirements.txt file.
Want to ship all your app and all it's dependencies in a single file, sweet as, use the shade plugin, or maybe the capsule plugin. etc. Etc.
The Python equivalent is, well, take your pick: https://docs.python-guide.org/shipping/freezing/#freezing-yo...
> Simple requirements.txt file
I've never had a JVM dependency break because I upgraded Maven. I have had that exact issue with Python - we're using Apache Airflow, and anyone who installed the latest version of Pip hit an issue - Pip had changed an API that Python dependencies depended on, so Airflow could no longer be installed by that version of Pip, because it was written for the old API. Have fun googling "TypeError: 'module' object is not callable" to enjoy that rabbit hole.
Not to mention that it was impossible to generate a pipfile.lock for Airflow because its dependencies had rules demanding incompatible versions of dependencies (one wanted <= 7.0.0, the other wanted > 7.0.0) - in Maven I can easily exclude one of the two competing dependencies.
Oh, and then there was the time that a Python dependency used by Airflow required me to set an env var to handle a licensing issue. Can't even do that in Maven, which strikes me as a feature.
pip3 install apache-airflow
RuntimeError: By default one of Airflow's dependencies installs a GPL dependency (unidecode). To avoid this dependency set SLUGIFY_USES_TEXT_UNIDECODE=yes in your environment when you install or upgrade Airflow. To force installing the GPL version set AIRFLOW_GPL_UNIDECODE
Lastly, Gradle is being used quite a bit, but Maven is still pretty dominant. It really depends on if you want a purely declarative build tool like Maven, or one you can drop scripts into like Gradle. I prefer purely declarative for reproducible buils.
Pip explicitly has no API (except for its command line interface), even the Pip namespace implicates this (pip._internal). So if some dependency is using a Pip "API" they're wrong.
Now, I am developing an application that needs to interface with Pip, in this case it may be appropriate to use the internal Pip API if you pin your Pip version in requirements.txt. However a library shouldn't do this ever.
BTW, I don't rate Python dependency management very high: most other languages have better solutions. However it is manageable.
https://github.com/apache/spark/blob/121f9338cefbb1c800fabfe...
At least half of that is figuring out what the classpath should be. And the worst part, it is all exposed to user, too -- when I was setting up Spark, I had to muck with class paths a lot.
So JVM has 3 big problems: (1) No single way to set up dependency path, each project is different; (2) No way to set up classpath in JVM language, has to use some other one (like bash); (3) Somehow, user has to fix it for many semin-advanced programs. No other ecosystem I know of is this insane! Python, node.js, even C++ are all better!
Okay, so, speaking for myself, we generally ship fat jars or capsules these Docker days. I can't speak as to why Apache Spark chose to do it that way, I don't run their project. But if I'm guessing I suspect it's so you can provide your own Hadoop dependencies as needed, as there's a bunch of varying distributions that people use (Cloudera, Hortonworks etc.) and they're trying to play nice with the existing Hadoop ecosystem - probably also to make it possible to extend it with any other jars that you want to use (although spark-shell has a nice feature where you provide additional dependencies on the command line, or within the shell, using Maven artifact coordinates)
For an example of what's involved using an uberjar https://docs.cyclopsgroup.org/jmxterm > java -jar jmxterm-1.0.0-uber.jar
That's it.
BTW, the uberjar equivalent in Python is something like PEX/shiv/zipapps.
looks like a custom one.. and non-relocatable, too -- if I wanted to install "jmxterm" to my home dir, I'd need to manually edit the script.
(and yes, this is totally "I am developer" issue, not just "I am a user" one -- the dev had to write the script, they had to debug it, and dev would be handling bug reports about inability to invoke from command line when installed to other location)
Funny how you had to go searching to find that script, when the installation instructions are simply - 1) download uberjar, 2) run uberjar. Also, you had to go searching to find a script 8 years old, in a project with a commit 20 days ago, to attempt to prove your point.
Surely you must realise you're grasping at this stage.
Your advice: forget whatever brought you to Python and start from scratch.
The articles arguments against this and for always using python -m pip don't convince me, personally. I never work on windows, so that's a mute point, and b/c I use direnv to automatically activate my env when cd'ing into the directory, I'm never not in the project environment. So basically, its impossible to not be using the correct pip version i.e. I have never once had this problem and I am constantly working on multiple python projects + code bases in various envs and versions.
And then python “just works”.
Does your text editor know that? Your colleague? How about your CI-system? What about the deployment story? When are things “unexpectedly” going to break because not everyone knew all these informally applied requirements?
Admit it: python is worst in class here. Almost no other popular language or platform forces it’s users into taking these absurd, error-prone steps, just to have basic shit “just working”.
Take a look at Node: npm may be a mess for other reasons, but boooy do you have clean per project dependencies. Look at Cargo for something even better. Clojure has Lein. The new dotnet CLI is slick too.
At this Python just plain sucks. It’s about time you guys start at least admitting it, or else it will never be fixed.
As far as the CI-system knowing, there is not much difference in installing a particular version of rust/cargo say in a docker container and having a particular version of python installed in a container, so I don't really know what you are getting at.
Could it be better? Sure, I do some programming in rust and IMHO the best build tool I've ever used is cargo. But compared to some of the other stuff you mentioned which I've used on projects as well (npm, lein specifically) I would say python is on the same level, and I prefer python over them but that's probably just my bad experience with package-lock/caching and long jvm load times speaking.
Whenever a discussion of this comes up, it seems there are always some "just use xyz" patterns, and usually 3 or 4 xyz or some very new thing someone just released and it always seems contradicts python official documentation. For every python packing question, there are 33 answers. I guess the irony is that there is more than one way to do it.
Use poetry, use prose, use pip, use venv, use vidtualenv, use conda, use --user, pipenv, pipfile, pipy,pipx,anaconda,miniconda.
It really is terrible. Npm gets a lot of flack for having vulnerable packages but do you see how many python fanatics push pip freeze? It allows them to ignore security vulnerabilities. I'm scared of upgrading any dependencies in my python projects because any update is virtually guaranteed to break. I can upgrade npm all day long and rarely encounter an issue. The ecosystem is so radically different.
Go doesn't have package management, it's just list of git repositories and revisions that should be checked out and compiled with your code.
In fact it is encouraged to have as little dependencies as possible.
- great concurrency (https://trio.readthedocs.io)
- bad package management (it just is. though Poetry's making a dent.)
- tooling? I'm not sure what you mean here.
- docs are ok not great not terrible
- types are rolling out rapidly across the ecosystem
- stdlib is too big but PyPI is right there so who cares
- ok to distribute? pyinstaller and shiv are alright.
He means linters, debuggers, refactoring libs, and other such things...
/s