Anyone who puts a local model on a GPU understands how much heat they produce.
Its amusing watching cloud AI users struggle to grasp even direct impacts of their use.
That's if you don't count hardware amortization - which is 80% of the cost...
That's not OPEX, that's CAPEX and tangible assets are depreicated not amortized.
The cost of electricity is (almost) negligible compared to the cost of having GPUs sitting idle. Electricity from fuel cells can be twice as expensive, but even then, it represents only about an extra ~15% of the amortization cost associated with keeping the GPU hardware idle.
That's not an operating cost, it's capital cost.
Yes, but given the 3-5 year lifespan of the server infra capex, the separation becomes less meaningful. Better to count the entire TCO or discounted cashflow accounting for the rapid asset depreciation.
There is no way it's 3% if they're getting electricity from the grid, even if they get their cooling from somewhere else (eg lake/river water or outside air in winter). Median DCs are anywhere from 25-40% (depending on the jurisdiction's electricity costs and how much cooling they need). Hypercaler datacenters tend to be even more dense than the median traditional ones.
That seems surprising - source? What's the other 97%? Does this include ops staff?
Without having done any research at this point I would assume it's mostly the amortized cost of hardware - GPUs, memory etc
That's capex not opex!
The stuff inside the buildings (servers, switches, storage, cooling, etc) basically the stuff using the electricity and salaries/sub contractors (in addition to some techs handling the servers think security, cleaning, storage management, receiving shipments of new hardware, etc). Some taxes (depends a lot where)
I saw a breakdown (don't have on hand sorry) where server depreciation was by far the majority. Power was the largest cash opex, probably matching your intuition.