I'm tempted to say it's not.

From experience compiling C++ a build can easily take 10 times longer depending on the optimizations flags that are enabled. The bulk of the time is spent in deep optimizations that may be architecture and CPU specific.

Wouldn't be surprised if compiling on a different architecture is multiple times faster because the compiler is not as optimized or doesn't have the same default flags.

It's fair in the real world, since that's what people will mainly be doing. But probably unfair from a benchmark perspective.

If the compiled application runs faster or similarly fast I think it's fair.

The point is to develop and run binaries. If X86 with SSE4 takes a lot more CPU time to compile binaries with acceptable performances compared to ARM, it's a win for the ARM architecture.

Great question.

I would assume there's some skew, but then again it's still a valid comparison even if the results were not to be generalized.

To me it really sounds like apples to oranges comparison.

Any test should compare same inputs to same outputs.