I feel like most of these problems just disappear if people would follow the same naming scheme sqlite3 uses. Just put the major version in the name and most tools just work out of the box with multiple versions.
Instead, we jump through hoops with our hair on fire to manage complexity.
People.
I'm an advocate for this style of library/dependency development, unfortunately in my experience the average dependency doesn't have the discipline to pull it off.
Code changes, and it's kinda silly to expect interfaces to be locked in place as that'll stifle development for even small-ish features. Does that mean every minor version or commit will change fundamental or large parts of the codebase? Probably not, but it's a sliding scale and people seriously need to find something better to do than writing Yet Another Python Package Manager.
I use the term "we" loosely here ofc in the context of this mini-rant.
If it really can't be done then you aren't really shipping a library that is meant to be depended on.
Imagine trying to do things like import pyarrow14, then if that failed try pyarrow13, etc. Additionally, python doesn't have a good way of saying "I need one of the following different libraries as a dependency"