> I hadn't considered it as a build speed optimization, though that seems somewhat obvious in hindsight.

Some build systems like cmake already support unity builds, as this is a popular strategy to speed up builds.

Nevertheless, if speed is the main concern them it's preferable to just use a build cache like ccache, and modularize a project appropriately.