It's, from my understanding, a little bit of both. There's a failure rate of GPUs and fans. There's also changing in standards like PCIe and software stacks.

LLM inference is mainly memory bandwidth constrained so I think it's highly likely that a company will create silicon with just an insane number of memory chips and less compute. These ASICs will probably do the same thing the crypto ASICs did.

If we look back 1 decade, no one uses a GTX 950 for anything.