Nvidia H100 GPU Shipments by Customer | Hacker News Reader

Nvidia H100 GPU Shipments by Customer

threads.net

28 pointsahiknsr24 comments

"Nvidia spends around $3,320 to manufacture a single H100 unit, which is then sold to end customers for a price ranging from $25,000 to $30,000." https://www.techspot.com/news/99839-nvidia-garners-reported-...

I'm sure a customer ordering 150.000 will get a discount, but how much? (150.000 x 25.000 = 3.75 billion)

Also consider that the $3,320 figure appears to be an estimate of the cost to manufacture an H100, which doesn’t encompass all of Nvidia’s COGS (R&D, warranty reserve, etc.)

agreed there probably is a bulk discount of some sort, but Nvidia has them by the balls because there aren't really other options, so they pressure to give good bulk discounts would be lessened a bit

Not only are there no other options, but Nvidia can't make them fast enough, so even more reason to not really bother with significant bulk discounts.

This makes sense, let's say that R&D per unit is a similar amount, that would totally explain how Nvidia is able to generate insane 50%+ net profit margins (74% gross margin), higher than FAANG and unheard of for hardware companies [1][2]

  [1] https://nvidianews.nvidia.com/news/nvidia-announces-financial-results-for-third-quarter-fiscal-2024
  [2] https://www.reddit.com/r/Infographics/comments/1825y3f/nvidias_q3_earnings_visualized/

It's seems fourth (at best) hand info, the source being a tweet by an analyst referencing another analyst.

NV does not manufacture chips, wonder if charitably they meant that they pay this much to the manufacturer? In that case this would be their price, I guess in a way "manufacturing cost" but quite misleading since the chip manufacturer probably has big margins.

Incredible that Meta has 1.5x the amount as Google and Amazon combined. For all the criticisms of Zuckerberg, Meta is an insanely strong and well-aligned organization, and their open-source contributions are unparalleled.

Google has TPU, might not need as many as others.

Likewise Tesla are at the bottom of the list but already have a significant investment towards their in-house silicon.

Buying H100's is almost necessary if only to assure you're not compute starved.

Motion is not necessarily progress. (Picture of a rocking horse.)

FOSS is all well-and-good, but that doesn't directly translate into profits.

Except it's based on a vague, "build it and they will spend" wing and a prayer cargo cult rather than fulfilling any specific, pressing need.

Morale is low as there are round after round of layoffs.

Amazon has their own in-house architectures that are designed for high performance AI training and execution.

So maybe that's why they don't have so much in the way of stuff from NVIDIA?

How would this 'Omdia' know who got how many GPUs?

I see many Chinese companies there. Wasn't nvidia not allow to sell this kind of gpus to China?

That ban only took effect around Thanksgiving (late November 2023).

It’ll be interesting to see how long these GPUs are in use, AND what the secondary market will look like in 5 years. Will NVidia shaft Microsoft and Meta like they shaft consumers re: driver support and other artificial limitations/planned obsolescence.

Theoretically Microsoft for example just needs to train one or two models and then retrain every so often, so I’m skeptical that NVidia will continue to knock it out of the park from a sales perspective. I wouldn’t be caught dead holding NVDA stock right now.

There is no source on this info, which would be commercial in confidence at the very least.

Also: how come Chinese companies are on this list!? I thought H100 exports to China were banned by the Biden administration!

Possibly things that were shipped before the cut-off date?

Ban took effect a couple weeks ago

Meta committed to spend $20 B per year on servers for 2023 and 2024. A large fraction of that is going to Nvidia for AI-powered, OCP-derived custom boxes that accept H100's.

The hilarious part is Zuck can't articulate a strategy for this spending at all.

*Who did NVIDIA sell its H100 to that isn't classified?

As I understand a H100 (SXM) gets you about 4 Petaflops on FP8. So 150k H100 is about 600 Exaflops on FP8. That is huge. Maybe enough to get AGI?

A can of coke is a few inches. 150K cans of coke is huge. That's enough to get to the moon?

Well, if I could bottle up my feelings about AI hype, they'd fizz over like a can of coke.

Maybe, if you already reached the atmosphere with it. Obviously, current AI tech scales with compute. It "may" be the case, that just more compute is enough to reach the moon..

Reply on news.ycombinator.com