Nvidia has insane margins so it isn't hard* for a well-funded player to beat them in FLOPS/$. The hard part is the compiler toolchains, the libraries, the documentation, getting models to run fast.

* Everything is relative. It's hard to make even a simple microcontroller - but also it isn't hard, you know?