It’s definitely surprising that macOS does this, but it makes me also wonder if the authors considered just modifying the binary so that the dynamic linker could find the library: https://www.joyfulbikeshedding.com/blog/2021-01-13-alternati...
I hope I make it in the preamble clear that this is bad and one should not have to deal with this – but it happens in practice and I hope such a summary is useful.
For posterity: if you want to wrap / use a C library in Python, you should go for CFFI (Cython works too and is overall faster, but has other downsides). This PyCon US video is a great up-to-date summary: https://www.youtube.com/watch?v=gROGDQakzas
I've tried it by adding the DYLD_LIBRARY_PATH to os.environ before calling ctypes.LibraryLoader.LoadLibrary (https://docs.python.org/3/library/ctypes.html#ctypes.Library...) and it didn't work. I suspect ctypes gets somehow initialized much sooner and adding environment variables in your apps doesn't help.
TBH I didn't research it further, since the problems of the post are more general and it can happen that you trip into them regardless of runtime.
I believe you should just be able to replace the library name with an absolute path to the library, and remove the need for the lookup at all?
If this were a production issue, I would probably fork the driver and do what you're suggesting (it's not like it's actively maintained or something :|).
Then the fix is a single if statement. Seems worth it to me.
One concrete thing that pops to my mind is that Cython doesn't support Py_LIMITED_API which means that you need to ship a lot more binary wheels. At least the issue is still open (https://github.com/cython/cython/issues/2542) and Cython projects IME need new wheels for each minor Python release. Compare that to cffi projects that (musl & pypy aside) only have to ship wheels for one Python version / architecture: https://pypi.org/project/argon2-cffi-bindings/#files
rpath doesn't just apply to executable binaries, it applies to all shared objects including libraries. So, what you can do is modify the dyld that the python/ctypes module will eventually dlopen(). You should be able to find it by rooting around in the files installed by the module in question.
rpath is evaluated not just at binary execution, but also upon any dynamic linking -- including dlopen(). So it will work for both ctypes and CFFI.
Happy to provide further pointers if you need!