I have to imagine that the vast majority of the CI time is spent rendering HTML and executing javascript on a wide variety of hardware and software combinations, and has nothing to do with compiling. Even the simplest possible support matrix would look like this:

MacOSX

===> Monterray, Big Sur, Catalina

===> x86, Aarch64

Windows

===> 10, 11

===> x64, Aarch64

===> Intel CPU $(flavors), AMD CPU $(flavors)

===> Nvidia GPU $(flavors), AMD GPU $(flavors), Intel GPU $(flavors)

And that's excluding Linux where the combinations get even crazier

The builds take about 10-20 minutes wallclock time, depending on platform.

Each test job generally takes at least that long, and this is after being split into many, many, many jobs to make it go faster.

Compiling is actually is tiny part of what runs on CI.