Why would they be shot? Unlike the consumer cards that are basically factory overclocked to look good on benchmarks, the datacenter GPUs are designed to run at full tilt 24/7 and survive for years.
From an annualized number on the llama 3 training report. would be interesting to see if we have a better idea given that we're already on rubin.
Also consider mixing in "was dropped during shipping" or "was stored improperly".
Why would hardware failure raise exponential with linear scale? Doesn't make sense
That's what you get from a constant failure rate per unit of time.
It's saying that hardware fails in the same way that Carbon-14 dating works.
Most if not all hardware failure on my side is def not memoryless besides the one lightning damage thingy but thats not hardwares fault.