If you can't use a "reasonably priced model" GPU for "a few years", I'm really confused as to what you're doing. I know people still using 1080's and 1080Ti's and playing pretty much anything they want to, and I only just upgraded from a 2070 Super to a 7800 XT (with 16GB of RAM on it, even) this summer.

Margins on datacenter GPUs will probably always be better than consumer. As long as thats the case they need to segment to stop datacenters from using consumer products so I've got a feeling that you will never be able to buy a consumer nvidia product with a reasonable amount of RAM. Maybe intel will release one to get some hype for their gpu line?

I’m just thinking that maybe trading a decent used sedan for a slightly shinier ARPG isn’t the wisest move I could be making.

I don't think it would work as well as a 4090 that has almost as many shader cores, and quite a few more cuda cores than h100.

Seems the pros of an h100 over a 4090 are: much higher vram, much faster vram, technologies like nvlink available, and a focus on lower precision performance more useful for ML (as opposed to 4090s focus on fp32).

There isn't infinite production/packaging capability, and they're going to prioritize the customers willing to pay more for the chips they get out of a wafer. Another aspect is that the chips are different between compute and consumer, as opposed to something like a Zen chip where it can be used in either Epyc or Ryzen.