I was hoping part of this delay was due to people arguing lock files are poor engineering to begin with, but alas, no mention of that. I guess we've just given up on any kind of package version flexibility.
pyproject.toml describes the supported dependency versions. Those dependencies are then resolved to some specific versions, and the output of that resolution is the lock file. This allows someone else to install the same dependencies in a reproducible way. It doesn't prevent someone resolving pyproject.toml to a different set of dependency versions.
If you are building a library, downstream users of your library won't use your lockfile. Lockfiles can still be useful for a library: one can use multiple lockfiles to try to validate its dependency specifications. For example you might generate a lockfile using minimum-supported-versions of all dependencies and then run your test suite against that, in addition to running the test suite against the default set of resolved dependencies.
Presumably because decades of experience has demonstrated that humans are extremely bad at maintaining compatibility between releases and dealing with fallout from badly specified package versions is probably second only to NULL in terms of engineering time wasted?
Or possibly it's just because a lot of the Python ecosystem doesn't even try and follow semver and you have no guarantee that any two versions are compatible with each other without checking the changelog and sacrificing a small chicken...
Even if they try, semver can only ever be a suggestion of the possibility of compatibility at best because people are not oracles and they misjudge the effects of changes all the time.
Then there’s Python - which I dearly love - that uses something that looks exactly like SemVer, but isn’t. This is often confusing for newcomers, especially because most libraries use SemVer.
Singling out two behemoths designed by committee to demonstrate SemVer's shortcomings seems misleading.
This is one of those things that becomes clearly untrue if you try to apply any amount of rigor to the definition of "just fine".
I have yet to see a single major Python project claiming to use SemVer that didn't occasionally unintentionally violate the promise made by the versioning because you cannot always easily predict whether a change will be incompatible, and, even if you could, people still regularly just make mistakes because people are not perfect. Versioning is hard. Keeping promises about not breaking things is even harder.
That may be good enough for things that don't matter, but it's not good enough for things that matter a lot.
Maybe I'm being too pedantic here, but semver for applications is always going to be broken and driven by marketing. SemVer, for my money, is only applicable realistically to libraries.
To say that you don’t have to upgrade is true, but it always comes at a price.
Whether someone else or I am the problem doesn’t matter to my customers at the end of the day, if I’m unable to ship a feature I’m at fault.
Well, yeah, it's reasonable people flame you there. What is the difference between,
- zlib-1 v 23
- zlib 1.2.3
except that automatic updates and correlation are harder for the first approach. It also will likely make typosquatting so much more fun and require namespacing at the very least (to avoid someone publishing, e. G., zlib-3 when official projects only cover zlib-{1,2}.
A little discipline and commitment to backwards compatibility and it isn’t too hard, really?
Literally everyone posting here is using a system built on compatible interfaces. Stable DLLs on Windows, dylib framework versions on OSX, ELF SOversioning on Linux.
It's clearly not impossible, just a question of priorities and effort, and that makes it a policy decision to do it or not. And I'll lean towards we've been shifting that policy too far in the wrong direction.
I look forward to you demonstrating your tool that can check if two python packages are compatible with each other, maybe you can solve the halting problem when you're done with that?
Also, I didn't claim any tool would give a perfect answer on compatibility. They don't for ELF libraries either, they just catch most problems, especially accidental ones. The goal is 99.9%, not 100%. Just being unable to solve a problem perfectly doesn't mean you should give up without trying.
You could call that success but I think it’s just an extra layer of cruft.
Microsoft's famously terrifyingly large amount of engineering work that goes into maintaining backwards compatibility to allow you to run Windows 3.1 software on Windows 11 is certainly impressive, but maybe also is the exception that proves the rule.
> Just being unable to solve a problem perfectly doesn't mean you should give up without trying.
Currently no one can solve that problem at all, let alone imperfectly. If you can, I'd gladly sponsor your project, since it would make my life a lot easier.