Today, if you are interested in a reproducible build environment, you will use pyenv to locally install an interpreter and stdlib, and pipenv to locally (to your project) install libraries and dependencies.
Rust is doing some interesting things with Rust 2018 Edition, where there is a complete system harmonization point. If it takes off there, I'd expect other languages to pick it up.
My question though is a bit different: Can Python catch up?
I know - strange question, but momentum is like that. It's often hard to see something going faster than you until it passes you. Perl is probably the best example, where it was way out front, and the PHP, which was way behind, but going mach 10, passed it quickly in the web space. Perl never caught up, and has lost significant relevance compared to the stature it used to command.
Consider; the reason for installing dependencies locally to your project is that it allows you to be precise about what library version you want. For example in Rust, we have Cargo.toml and I can have dependencies like primal = "0.2". I hate this model. It takes us right back to the old days of projects shipping whatever version of a library they want and never updating it. For small projects this is a constant source of security issues and code stagnation.
I groan internally every time I see a project I'm interested in is written in node. Not because I dislike the language, but because I know when I download and build it there are going to be a dozen outdated dependencies hardcoded in the config file, some with "severe" security warnings flagged by npm.
I advocate letting library developers and distribution maintainers do their jobs. Most of the time new versions of libraries are supposed to be backwards compatible. When they're not the distribution can ship multiple versions of a library and make sure they all have security updates backported.
There are cases where managing all your own dependencies is important (like closed source web applications), but I think those are exceptions. Dependencies were supposed to be a solved problem.
It evolves constantly. Python packaging then and now has massively changed (largely without breaking compatibility, mind you). It's not perfect, but it is much better than it was in 2005.
I know pythons big thing is backwards compatibility, and breaking things is almost the biggest sin in that community (python2...), and maybe I’m ignorant, but I just don’t see how implementing a single standard package manager with a standard manifest and lockfile would break anybodies anything. All they have to do is make it so pip still works without it, and if they want to see how to do that, just look at NPM, who before this year was notorious for errant package versions due to the lack of lockfiles, but now they have it and I think pretty much everyone was happy about it. They literally could just port bundler to Python and call it a day, the code is open source, they can look at how it works, and it’s one of the most well regarded package managers out there, I’ve maybe run into 2 issues ever with it. The only other package manager I see get more praise is cargo, which they could (should) also look at for inspiration.
I know things have been improving. When I was using Python daily, the standard was still punching in some incantations to start of Virtual Env (which still confuses the hell out of me to this day) and piping the contents of a requirements.txt file into pip (with I think at least some level of version locking). But, from what I’ve gathered is there are some solutions being built to bring pip out of the early 2000s, but they’re quite fragmented efforts, and they all fall short in different places.
Python is a great language, pip and all the bullshit that comes with it is terrible.
Now that I’ve wrote that out, I’m wondering if packages can even pin their deps required versions. I’ve only published one python package years ago, and I can’t remember if that’s possible. Which I shouldn’t have to even worry about in a modern package management system.
I know the above example wouldn’t be possible in any of the systems I mentioned above, because the aside from installing packages, they ensure that the versions of every package are compatible. That’s why lockfiles are so important, assuming you’re downloading decent packages, the package manager can assure you that they’re all going to work together, also allowing you to update without having to worry about some random deeply nested dep isn’t going to break some other package when it gets updated, making updating (usually) a breeze.
In pips current state, it seems more or less like a slightly more strict NPM before yarn made them get their shit together (still vastly prefer yarn). The only difference is your packages are wherever Venv puts them instead in a “pip_modules” folder in the project, which is at least better than a global free-for-all. Though, that also might be less of a headache than dealing with Venv, I hope it’s gotten better since I’ve used it, because that was always such an annoyance.
> Now that I’ve wrote that out, I’m wondering if packages can even pin their deps required versions. I’ve only published one python package years ago, and I can’t remember if that’s possible.
It is.