Nvidia's fat margins are a worrying sign of its market power | Hacker News Reader

Nvidia's fat margins are a worrying sign of its market power

lightreading.com

47 pointsmgh237 comments

Open articleView on HN

Good for Nvidia. They solo explored and invested in parallel computing hoping it would pay off and it did.

The fact their margins are so high is a measure of how much their competitors thought that Nvidia was wrong.

The funny thing to me is that crypto crashed immediately before LLMs exploded, WTF!?!

A timeline where they both simultaneously happen or LLMs come first would have changed the future quite significantly.

Let the crypto miners run LLMs to generate crypto propaganda and spam it on the internet.

The gap between Nvidia and AMD isn't that big. At the moment if you want to be involved with cutting edge AI you have to go with Nvidia. There is no evidence at all what the situation will be like in 5 years though.

Not to mention that the clock is ticking until other market players start really responding. The situation is very fluid and Nvidia was ready for it, but they haven't discovered a magic formula that lets them overcome commodity economics.

Sure, the hardware gap isn't that big. The problem is NVidia controls the software side with CUDA.

Yes, but unlike behemoths like Broadcom who have successfully locked down certain types of software through patent-fu, Nvidia does not (currently) have a way to stop AMD from creating their own software.

This means that with enough incentives - and there are plenty right now - the competition can reasonably catch up in the next 2-3 years.

> the competition can reasonably catch up in the next 2-3 years.

Then how long after that for similar levels of adoption among devs?

The situation today is evidence for the situation in 5 years, and yes they discovered a magic formula: the integration of hardware and software.

> Nvidia is now the tech equivalent of an oil producer during an energy shortage, seemingly able to charge what it wants and pocket outrageous profits

The other "oil producer", AMD, does not make a good competition. All cheap graphics cards are Nvidia. This is the same situation like with Windows: when everyone use it at home (pirated), or school (free), they will use it also at work ($$$).

> All cheap graphics cards are Nvidia

If you watched any serious review last year you'll notice the AMD GPUs were consistently better value than their Nvidia counterparts. Though the 7000 generation was still very disappointing regardless.

Steam hardware survey to the rescue.

If you want to use ROCm, then you better pay up for those datacenter cards. The consumer GPUs from Nvidia are cheap in comparison.

Consumer cards work too, except for the RDNA 1 generation.

That was a similar business model of Sun Microsystems and SunOS/Solaris operating system, except it failed to be a business desktop OS for the workplace... and then Oracle came along and murdered any chance of that idea taking effect at a later date.

Aren’t their margins a product of producing better products?

Market power implies it’s because nvidia is using methods other than better products to keep its prices high.

That's because they are. Ever since the mining crash Nvidia has been selling all its GPUs for exorbitant prices, much to the detriment of people who need the VRAM for machine learning. Oh well, guess you're buying the (checks notes) $13,000 A100, just because you need 3.3 times the VRAM of the $1600 4090 (which could be $5280 instead, if price corresponds exactly to VRAM).

At the risk of sounding like a broken record: Once others get their software shit together, they can have some of those profits. Nvidia has had a competent software game for a decade.

They still haven't gotten their hardware shit together. As a consumer I can't buy a PCIe card that slots into my system with more than 24GB of VRAM, whether it's from Nvidia, AMD, or Intel. Nvidia's prices will continue to be high while there is literally no competition.

I guess before datacenter class cards can trickle down to the consumer, they do have to get their software shit together first.

https://www.bhphotovideo.com/c/product/1753962-REG/pny_vcnrt...

NVIDIA and AMD both have workstation PCIe cards with more than 24GB or VRAM (W7900 from AMD and RTX 6000 Ada both have 48GB).

They just want more money.

Cool. Where can I buy one?

If I go to the page for the W7900, I can download the datasheet. I can look at the marketing pages. I can look at the specifications and download drivers. But I can't buy the card.

If I look for one on eBay, they start at $4600, which is still far too expensive for 48GB. At that point, just buy an Apple Silicon machine and you can get far more memory than that for even cheaper. (M3 Max MacBook Pro with 96GB memory is just $4,000!)

Alright, let's try the RTX A6000! Oop, it's $4,000. Cheaper than AMD, but wow! Oh, and I can only find these prices on eBay, you can't buy them direct from Nvidia as a consumer.

Just checking here but are you comparing a macbook with 96GB of machine RAM to a GPU with 48GB of on-card RAM?

Because if so you can get a good pc with 96GB of machine ram for significantly less than $4000.

Unless the claim is that an M3 Max MacBook has equivalent GPU type compute power as a dedicated GPU?

It's probably because of the unified memory model - the GPU on the macbook can access the memory as if it was dedicated.

Interesting, i didn't know this existed.

I'm comparing the cost of a GPU with 48GB VRAM to the cost of an Apple Silicon machine with at least 48GB of Unified Memory, yes. (The ASi machine having 96GB of memory instead is just because it's still cheaper than the card, so why not.)

A normal PC with 96GB of regular RAM wouldn't allow you to use that RAM for GPU workloads, at least not with a dedicated GPU.

As stated above, i didn't know this existed.

Would an ASi machine and Dedicated GPU with the same amount of respective RAM have the same performance compute-wise?

> As stated above, i didn't know this existed.

My comment was made 10 hours before that reply, so I didn't have that context. Sorry.

> Would an ASi machine and Dedicated GPU with the same amount of respective RAM have the same performance compute-wise?

Not sure, but performance comes after making sure you have enough VRAM to run your workload in the first place. I doubt an ASi chip would beat most datacenter-class training cards, but if a consumer just wants to train a model locally without much regard for getting the best possible performance, there is more value-for-money.

> My comment was made 10 hours before that reply, so I didn't have that context. Sorry.

Sorry, i phrased that poorly, i was referencing my own comment, not because i thought you had ignored it but because i had just replied to the other comment.

It probably shouldn't have reference the other reply.

> Not sure, but performance comes after making sure you have enough VRAM to run your workload in the first place. I doubt an ASi chip would beat most datacenter-class training cards, but if a consumer just wants to train a model locally without much regard for getting the best possible performance, there is more value-for-money.

That mostly makes sense.

Though personally,if i'm dropping car money on hardware to run a model then i'd really want to know the type of difference in power, are we talking about the ASi taking twice as long or are we talking orders of magnitude?

Not a question for you specifically, just a thought.

I'll see if i can find any benchmarks for the differences in compute.

Thanks

> are we talking about the ASi taking twice as long or are we talking orders of magnitude?

I'm not aware of any benchmarks comparing Apple Silicon to datacenter-class chips, unfortunately.

NVidia primarily doesn't sale to consumers. You'll have to get from 3rd-party like PNY. It is costly because they have customers lining up to pay that price. If demand goes down price will follow.

And same applies to AMD. A reputable retailer selling W7900s: https://www.bhphotovideo.com/c/product/1765536-REG/amd_100_3...

Does the price of whatever SaaS you're working on correspond exactly to the RAM usage of a customer?

Well, my (totally made-up) calculation conservatively assumes that multiplying the VRAM multiplies the price by the same amount. In practice I don't think it's even that expensive to add more VRAM, since most of the rest of the card should stay the same?

Market power just means having a negotiating position that allows you to determine prices, regardless of how that negotiating position is obtained. It can either come from illicit means (say cartels for example), or from true innovation and investing into the right things.

I think that both the customers of Nvidia as well as the competition are keen on replacing it (given the scale that many of the big customers work at, I'm sure they can afford the engineers to migrate the LLM models to ROCm if it were a viable alternative), but it seems that they cannot manage to make replacements fast enough.

> The most interesting detail is that Nvidia managed to double sales without incurring additional costs

market power also comes from a material log jam because we can't make GPUs fast enough to satisfy demand, so the wealthiest customers choose the price because the GPUs built essentially go to auction.

"In economics, market power refers to the ability of a firm to influence the price at which it sells a product or service by manipulating either the supply or demand of the product or service to increase economic profit.[1] In other words, market power occurs if a firm does not face a perfectly elastic demand curve and can set its price (P) above marginal cost (MC) without losing revenue."

https://en.wikipedia.org/wiki/Market_power

Sigh. I mean may be HN want to take a look at [1] NVIDIA Profit Margin 2010-2023 | NVDA, which include Net Profit Margin, Gross Margin as well as Operating Margin and make up their own mind. People tends to have very short memory, so many may also want to Google what happened to Nvidia in 2022 and what it did to their stock price.

[1] https://www.macrotrends.net/stocks/charts/NVDA/nvidia/profit...

(August 2023)

Reply on news.ycombinator.com