Yeah it would suck, but if Ford F150s were selling for $100,000 because drag racers were buying them up, I don’t think I’d mind.
We often have used GPU's for around the price range you're looking for. They won't be fantastic, but probably an upgrade for a 9 year old machine.
Edit: I should also add, a lot of the really small scale shops also don't list 100% of their inventory online, with the benefit that it doesn't get scalped up like everything else.
Modern computers have allowed these 'forever existing' technologies to scale to our population growth. Try building freeways without modern computers, or performing modern medical interventions without them.
Probably the most critical usage of GPUs is scientific simulations for biology/medicine, climate, space, etc.