Cuda is a proprietary Nvidia framework. You don't need cuda support to train deep learning models on TensorFlow. ROCm also works - and is built by AMD and is open-source.
Unless AMD can provide a toolchain which takes cuda code and generates whatever it takes to run it with performance parity to Nvidia cards, it'll never take off.
ROCm is a decade too late to simply coexist with cuda and battle for market/mind share. Unless they can offer a massive perf/cost advantage, there is no incentive for anyone to invest in porting their code.
They have this. It's called HIP.
"HIP is designed to ease the porting of existing CUDA code into the HIP environment."
Not even close an drop-in swappable alternative.
Khronos did a big mistake to keep pushing C as the only compute language for so long, and it remains to be seen how much adoption SPIR-V is able to actually take from PTX.
So unless HIP actually matches PTX, it is a non-starter.
I guess I'm staying with NVIDIA with developer laptops for a long time, and maybe using AMD for ultrabook.
So if HIP will only work with LLVM, that is also a non-starter.
https://github.com/nixos-rocm/nixos-rocm
The last time we purchased GPUs, we bought two Radeon VIIs besides RTX 5000s. For training transformers in Tensorflow and PyTorch, they were a steal, because the training is only marginally slower than the RTXes at a fraction of the price.
For LSTMs, we have found performance regressions compared to NVIDIA, but it's largely because it was hard to hit the optimized MIOpen code paths.