Yes, building AOSP requires a fairly powerful machine (at least to do it quickly): https://source.android.com/setup/build/requirements. It's definitely possible to do on a local machine with decent specs though.
Now granted, those were heavyweight specs when Android came out in 2007, but I'd figure about half of us probably have a similar box sitting around today, and the other half would just need to beef one up with some additional RAM.