The "challenge" with installing scientific python library, especially numpy, is to make sure that it is built against the fast BLAS libraries (OpenBLAS, MKL or at least ATLAS). The default netlib BLAS library is too slow for any serious work.
But I have to admit I don't know the situation with the binary wheel packages.