Yep, bingo. There's no substitute for understanding exactly what's being emitted in the innermost loops of intensive operations.

I wonder whether any of this matters much in GPU-land though. I haven't kept up with the game, so the young hotshots that programmed AlphaGo might be doing all of that without resorting to low-level-ASM-type sorcery. I wouldn't be surprised if massive parallelism makes these types of gains a rounding error.