To be clear, basic IEEE-754 scalar operations are already not compatible across different CPU hardware vendors. At best, we can say that if you run the same code generated by the same compiler targeting the same hardware, you'll get identical results. But nothing else is guaranteed.

Just off the top of my head:

1. The chip that gave us the IEEE-754 spec, the Intel 8087, internally worked with 80-bit extended precision floats that nothing else supports. Things would get truncated to standards-compliant 64- or 32-bit when spilled to memory, but you'd have to actually do this between every operation to get standard[1] rounding behavior. At least until the 80387 which let you set the internal precision.

2. Many, many RISC chips implemented a fused multiply-add (FMA) operation that could multiply and add faster than issuing separate instructions. Naturally the FMA unit would also result in fewer roundings. x86 did not have a widely supported FMA instruction until 2014[0].

In general, every floating point operation is going to have different rounding characteristics and that is the source of all floating-point hardware variance. Every time the number or order of operations changes, the output changes. In order to specify an "exactly and precisely compatible" calculation mode you have to freeze in place those operations, forever, across both silicon and compilers. This goes against the basic idea of a matmul accelerator: every time someone finds a way to multiply two matrices faster through parallelization, different tiling, or a different sequence of operations, that changes the rounding and numerical stability characteristics of the matmul, and now we need a new spec.

[0] To make matters worse, AMD shipped an incompatible FMA extension that was later removed in Zen 2!

[1] This is actually fairly tame in terms of "hardware vendors deviating from IEEE spec" - the Sony PS2 shipped with completely out-of-spec garbage floating point hardware that infamous tainted ports of games to other competing consoles.