> GPUs that have a 1-3 year lifespan
In 10 years GPUs will have a lifespan for 5-7 years. The rate of improvement on this front has been slowing down faster then CPU.
In 10 years GPUs will have a lifespan for 5-7 years. The rate of improvement on this front has been slowing down faster then CPU.
What is interesting is that it seems like the ever larger sums of money sloshing around are resulting in bigger, faster hype cycles. We are already seeing some companies face issues after blowback from adopting AI too fast.
(It might be too expensive to pay for LLM subscriptions when every device in your house is "thinking" all day long. A 3-5k Computer for a local llm might pay itself off after a year or two. )
The next frontier would be training directly with block floating point, where you have a shared exponent plus the two remaining bits. It's getting tight.
Maybe it is possible to have mini LoRA blocks where an n times n block is approximated by the outer product of two n sized vectors. For n = 4 the savings would be 50% less FLOPs and for n=8 the savings would be 75% less FLOPs.
The three year number was a surprisingly low figure sourced to some anonymous Google engineer. Most people were assuming at least 5 years and maybe more. BUT, Google then went on record to deny that the three year figure was accurate. They could have just ignored it, so it seems likely that three years is too low.
Now I read 1-3 years? Where did one year come from?
GPU lifespan is I suspect also affected by whether it's used for training or inference. Inference loads can be made very smooth and don't experience the kind of massive power drops and spikes that training can generate.
Perhaps the author confused "new GPU comes out" with "old GPU is obsolete and needs replacement"
I believe that lifespan range came from cryptocurrency mining experience, running the GPU at 100% load constantly until components failed.
2. While comprehensive studies were never done, some tech channels did some testing and found used GPUs to be generally reliable or easily repairable, when scamming was excluded. https://youtu.be/UFytB3bb1P8
You can keep a server running for 10-15 years, but usually you do that only when the server is in a good environment and has had a light load.
I said solid state components last decades. 10nm transistors have a thing for over 10 years now and other than manufacturer defect don't show any signs of wearing out from age.
> MTBFs for GPUs are about 5-10 years, and that's not about fans.
That sounds about the right time for a repaste.
> AWS and the other clouds have a 5-8 year depreciation calendar for computers.
Because the manufacturer warranties run out after that + it becomes cost efficient to upgrade to lower power technology. Not because the chips are physically broken.
> Most of the money is being spent on incredibly expensive GPUs that have a 1-3 year lifespan due to becoming obsolete quickly and wearing out under constant, high-intensity use.
So it isn’t entirely tied to the rate of obsolescence, these things apparently get worn down from the workloads.
In terms of performance improvement, it is slightly complicated, right? It turns out that it was possible to do ML training on existing GPGPU. Then there was spurt of improvement as they go after the low-hanging fruit for that application…
If we’re talking about what we might be left with after the bubble pops, the rate of obsolescence doesn’t seem that relevant anyway. The chips as they are after the pop will be usable for the next thing or not, it is hard to guess.