> Isn't this what /opt is for though?
Pretty much yes; the idea was "/opt all the things!" (well, before the all-the-things meme existed — it was a long time ago!) and use symlinks to make everything appear as if it was installed in the "regular" Unix places (so that we wouldn't have to patch/reconfigure every single program).
GoboLinux has a long and cool story (I still use it!) but that's the tl;dr version of the motivation!
We used to get problems like this when building ROS on homebrew where you'd have a bottled version of Gazebo (a robot simulator) which had been linked against specific versions of boost, python, opencv, whatever. And then a new version of one of those dependencies would come out, and something else in the tree would update ahead of Gazebo; now suddenly you've got Gazebo plugins which crash with ABI issues and you have to go spelunking to find out which binaries are linked to what and where the conflict is.
It's not by accident! In its original docs, Homebrew described itself as package management "the GoboLinux way".
Given how Homebrew has become this super-popular tool, its success makes me proud of Gobo's legacy (and a bit vindicated from all the people who told us "this model is crazy, it will never be usable!" :) )
> keeping all versions of everything is that you end up with problems when you're trying to build/use packages with a large pool of dependencies
Yes, it can be a pain! In recent years, we introduced Runner (https://gobolinux.org/runner.html) in GoboLinux as a way to address this kind of issue; a virtualization layer to present the expected dependencies at the right places.
Clever. Seems like it's essentially a way to "pause" the rolling distro at a known-good point, and then use that environment to build whatever high-level stuff it is with your set of frozen dependencies— this would definitely do the thing, and would have been very helpful for the ROS on Homebrew effort.
OTOH, how different is this from Debian Sid as the rolling release that is occasionally broken when a new low level dep comes in, with the supported releases of Debian, Ubuntu, and others as the shared pause-points? I think ultimately I find it easier putting the work of finding a set of interoperable versions of things onto my distro maintainer (and in exchange knowing that if I want the cutting edge I'll have to do my own backport or go to PPAs), rather than taking on that work myself and hoping for the best or risking getting stuck with a bunch of super-old stuff and no clear migration path forward.
I want to produce an executable asset that has everything necessary to run, regardless of the wacky local environment that it's being executed in. I have lost so much time trying to debug environment related issues because (at least in my experience), debugging them requires a deep system knowledge of the dependencies and how Foo tool/program/framework is expecting them to be.
1. Security patching in one place
2. Memory usage
3. Allowing a program to be improved without it needing to be recompiled
Most obviously without dynamic linking, the concept of an operating system upgrade wouldn't make any sense. But, users quite like clicking "Update" and then their programs get new features.