In most real code the high throughput of these sorts of operations means that something else is the limiting factor. And if multiplier throughput is limiting performance then you should be using SIMD or a GPU.
True. You can imagine how difficult it is for the hardware engineer designing and testing these things before production!
And it's quite sad because when you are faced with choosing between two ways to express something in the code, you can't predict how fast one or another option will run. You need to actually run both, preferrably in an environment close to the prod, and under similar load, to get accurate idea which one is more performant.
And the worst thing is, you most likely can't extract any useful general principle out of it, because any small perturbation in the problem will result in a code that is very similar yet has completely different latency/throughput characteristics.
The only saving grace is that modern computers are really incredibly fast, so layers upon layers of suboptimal code result in applications that mostly perform okay, with maybe some places where they perform egregiously slow.
"The best way to predict the future is to invent it" -- Alan Kay
> You need to actually run both
Always! If you're not measuring, you're not doing performance optimization. And if you think CPUs are bad: try benchmarking I/O.
Operating System (n) -- Mechanism designed specifically to prevent any meaningful performance measurement (every performance engineer ever)
If it's measurable and repeatable, it's not meaningful. If it's meaningful, it's not measurable or repeatable.
Pretty much.
Or put another way: an actual stop watch is a very meaningful performance measurement tool.
Imagine having to design electronics the way we design performant programs. Will this opamp survive the load? Who knows, let's build and try these five alternatives of the circuit and see which one of them will not blow. Oh, this one survived but it distorts the input signal horribly ("yeah, this one is fast, but it has multithreading correctness issues and reintroduction of locks makes it again about as slow"), what a shame. Back to the drawing board.
A pro would generally only run benchmarks if it's the only way to find out (or if it's easy), but isn't going to trust it unless there's a good explanation for the effects, or unless they actually just want to compare two very specific configurations rather than coming up with a general finding.