I upgraded my website recently and it's exciting! That being said, I admit my builds didn't get faster (they actually on average slowed down a bit). Hopefully that improves, but worth noting.
We're working on incremental builds which should help as well: https://github.com/withastro/roadmap/issues/1388