Your mediocre graphics card may or may not have CUDA support. If it doesn't, or it only supports an old version, then it's the same as not having a graphics card at all. And of course, Nvidia only.
If it supported, then the major difference is GPU memory, which limits the size of the network you can train. The newest models are faster than some 1-2 year old ones, but older hardware does the job fine.