There are a ton of subtleties in the build toolchain even within a single OS type, and these will lead to downstream frustrations with packages that just bundle pre-compiled versions of C/C++ libraries.
The conda approach treats the underlying libraries as first-class citizens in the package ecosystem; tracks their interdependencies; and most packages are built in such a way that they are relocatable on your filesystem and don't require system privileges to install. conda and the various packages in the conda universe (whether official ones from Anaconda or the community-built ones in conda-forge) all make it so that this baseline hard problem is mostly solved across all major OSes, and solved in a relatively consistent way.
You can think of conda as a cross-platform, cross-architecture, multi-language, userspace package manager. It grew into this because numerical Python's package ecosystem is so horribly complex and the users span such a huge set of install environments, that conda ended up having to be a generic rpm/brew/apt kind of thing.