Just in case other people who have AMD GPU and run Windows have the same needs as I have, that is, to train or run machine learning models, please checkout torch-directml and tensorflow-directml.
cuDNN being higher level offers more opportunity for compatibility without losing performance (i.e different implementations of kernels fine-tuned for optimal performance on AMD vs NVidia hardware), but the trouble is that so much of what frameworks like PyTorch do is based on custom kernels, not just cuDNN.
It seems the best bet for AMD would be a rock solid low level API (not a moving target) and support of high level optimizing ML compilers to reduce the level of effort for the framework (PyTorch, TensorFlow, JAX ...) vendors to provide framework-level support on top of that. Ultimately they'd need to work very closely with the framework vendors to provide this support, since they are the ones who would be benefiting from it.
It's odd how much of an afterthought ML support has seemed to be for AMD over the years... maybe the relative size of the consumer ML market vs graphics/gaming market didn't seem to make it worth their effort, but as NVidia has shown this is a path to gaining much more lucrative data center wins.