Related question, does anyone have experience with using the AMD MI100 for deep learning? With 32GB and a second hand price of ~1100 USD, it could be a good choice.
Allegedly they perform near an A100, so raw-compute wise, memory capacity wise, and memory bandwidth wise, they rock. As is typical for anyone not Nvidia, the software is still playing catch up. To be fair, Nvidia themselves takes nearly a year to build out all CUDA features for some of their cards - FP8 for example, is only recently become usable on a 4090.