I don't know when they introduced it, but I believe MacPorts will build the common variants of the more-used packages. So, if you install a package with the default variants, you'll get a binary download instead of building from source.

But indeed; fast SSDs, parallel compilation, and modern CPUs really help!