Have you tried out Spack? I used this quite a bit in academia to install the scientific sorts of libraries many Python packages depends on without needing to rely on the OS versions which were often out of date.