Four co's are hoarding billions worth of Nvidia GPU chips. Meta has 350K of them | Hacker News Reader

Four co's are hoarding billions worth of Nvidia GPU chips. Meta has 350K of them

sherwood.news

27 pointsSurfTokyo15 comments

To me, "hoard" comes with an implication of amassing these in secret and/or not making full use of them. If Meta is crowing about how many they have, and seemingly putting them to good use (or at least some use), I wouldn't call that hoarding.

I occasionally acquire half-decent hardware for stress testing. If I add a pair of Dell servers to the lab (or, heaven help me, anything from IBM), I don't feel like I'm hoarding. What I do feel, after spending $50-500K of someone else's money, is a responsibility to get their money's worth out of it.

I can understand, then, Tesla shareholder frustration with 12,000 of these being diverted to a different neighborhood in Muskville. Talk about other people's money.

That's an estimated $7-14 billion worth of H100s in Metas racks and they're using them to make models which they give away for free, with no clear plan for how that's ever going to be worth their while. I'm getting flashbacks to Meta acquiring Oculus a decade ago and still having absolutely no idea how to even make it break even, they just keep dumping endless billions into it in the hopes that it will eventually all be worth it, somehow.

I know people here like to imagine Zucks AI strategy as being 5D Chess, but in light of Oculus it's hard for me to see it as anything but desperately scrambling for a pivot to the next big thing after they hit peak Facebook.

I tend to agree. I'm thrilled with Mark Zuckerburg's strategy of releasing these open models, so I'm perfectly happy for Meta to own all these GPUs. I like what Meta is doing with them and I want this open weights approach to work, and it's at least plausible that Facebook/Meta benefits from high quality open weight models being available to everyone. But having said that, it's not at all clear to me how Meta is going to recoup their costs. It was great having Stability AI release their open weight models too, but it was never clear what their business model was either.

In the short run, it's great that we're getting these open models from Meta. In the long run, there needs to be a real viable strategy that makes this investment actually worthwhile, or these open releases will cease.

Hopefully the plan is as simple as it seems: plan is to keep online communications (which Meta is at the center of already) vibrant and creative, and to keep anyone else from coming to dominate & control that.

A pretty sensible hedge against existential risk.

They’re using them for everything from video streaming to training models. It’s probably one of the best utilized fleets out there, in terms of full utilization, and there’s direct correlation to revenue. So closer to a very tightly knit checkers play than 5D chess, really

Imagine the depreciation costs involved in this... Huge money sink.

To the contrary, this might be a good enough generation to hang your hat on for multiple reasons:

- TSMC is fickle. 5nm was a one-customer node for a few years, and Blackwell necessitated a large investment to get Nvidia on the latest TSMC tech. Die shrinks are getting expensive and less-certain; buying now at least guarantees you good software support on top-class hardware.

- The geopolitical situation threatening Taiwan is dire enough to force the hand of anyone that wants to guarantee GPU availability. Buying GPUs now when they're highly-available is a hedged-bet against a worse GPU shortage than ever seen before in the face of unprecedented territorial aggression.

- ...it's Nvidia. If the past is anything to go off of, these GPUs will get software support for a decade and be worth a pretty penny on the resale market. Worst case scenario, AI is outlawed by the next president and Meta pivots to Bitcoin mining or something. Hell, if they outlaw AI fast enough they could probably sell their Nvidia hardware at a profit.

Nvidia has the best hardware and software for deeplearning, but current use for it, and this huge demand was driven mostly about LLMs.

The workload done by LLMs can mostly be done by different hardware, that are more ASIC-like and efficient, see Grok for example.

If there aren't enough deeplearning uses for Nvidia hardware, meanwhile the big techs might not recognize on its balance sheet, its ability to generate money down the road will be much smaller.

I'd be willing to bet that in 10 years, H100s will be worth 0. It's a huge writedown.

I'm sure we'll do much more deep learning down the road, but the demand recently for this hardware has been too big, mostly because of hype, this will definitely have an impact on depreciation once the hypes fade off.

Even Meta, when they figure out they can do their own custom hardware to do their Instagram recommendations, they will talk directly to TSMC. The CAPEX on this is huge, so are the savings in case they find out a way to use custom hardware.

Google went through this route because the found out themselves that it's worth having their own hardware as they do a lot of AI. Given this, I believe that it's a huge risk to have H100s at the moment, so does Blackwell. It's a risky asset.

>I'd be willing to bet that in 10 years, H100s will be worth 0. It's a huge writedown.

There wouldn't be a writedown simply because no one is setting 10 years as the useful life for accounting purposes. Coreweave was about 5 years IIRC.

> Buying GPUs now when they're highly-available is a hedged-bet against a worse GPU shortage than ever seen before in the face of unprecedented territorial aggression.

This actually makes it occur to me that if you've been hoarding a huge amount of GPUs, it may actually be in your interest to then encourage the events that would lead to a subsequent GPU shortage.

Just think of how much e-waste this will be in a few years once it no longer makes sense to run them due to improved FLOPs per Watt of newer silicon.

Does Google/Alphabet also use the previous-generation A100 GPUs that the article says Microsoft is running, or has Google shifted fully to its own design?

Google has both H100 (and previous generations) and TPUs.

Different teams and projects use different accelerators.

In fact, I am a bit surprised to see Andreessen Horowitz mentioned in the fourth place, since Google has more than that number of H100.

You can hardly hoard things that will become obsolete in a few years.

Have they made any money running these?

Reply on news.ycombinator.com