If you were to design a GPU several years ago, people would probably tell you designing it for AI is crazy and cryptocurrencies and raw hashing power is where the money's at.

In the end, Nvidia kind of got lucky with their bet on AI. While CUDA has been a thing for a while, it was limited to mostly academic and workstation workloads. It wasn't until LLMs and things like Stable Diffusion when it really became popular.

Unfortunately for non-NV users, most of these AI workloads when run locally assume CUDA is present and it's a considerable effort to make them work on non-NV GPUs. It's fairly okay for common workloads like running basic LLMs and SD models, but as soon as you get into more advanced/obscure stuff the harder it gets without CUDA.