I mean if you're worried about high performance compute on cpus, then the argument also applies. That's why there so much hand written assembler with dynamic dispatch on CPU features...

It's just that gpus have no general purpose usage .. for ai, it's basically all about perf.

Of course some cross platform stuff can be made for modest acceleration, but the state of the art is always going to be out of reach.