I presume you compile C/C++ code. How does your file structure look like? Do you have a lot of files with little to no content? You can speed things up by merging files. I have managed to reduce the building speed by around 95% by doing such simple things.

Reducing the amount of unnecessary headers is also beneficial, but in comparison to merging files it barely makes a difference.