Nvidia RTX 5000 Ada 32GB Workstation GPU Review | Hacker News Reader

Nvidia RTX 5000 Ada 32GB Workstation GPU Review

servethehome.com

70 pointsrcarmo105 comments

Open articleView on HN

Very disappointing, much less cuda cores than in 4090, no nvlink (32G won't be enough anyway), price x3 to 4090.

>much less cuda cores than in 4090

"fewer", not "less".

While that's true, it feels a bit pedantic to correct parent like that.

What would be the right way to correct him?

I think the right way is, don't. This kind of "mistake" can not really result in any misunderstanding so there is not really any benefit gained from a correction. Correcting serves only to disrupt the flow of the conversation.

It benefits me, the non native english speaker passing by in the comments and who learnt something today.

I'm a native English speaker. I don't personally like corrections like this, and I would never criticise anyone for using the 'wrong' term. There is too much convention in English which isn't useful because it bleeds into pedantry and I think this is such a case. Using either word gets the job done cleanly, and at the bottom that's all that counts.

The problem is you take this as a critic, which is not.

This feels so wrong to post in this thread, but I’m going to bite. It’s ‘critique’, I believe you mean.

I’m sorry.

Edit: as someone who can’t even figure out the markdown to italicise right now, maybe I should have kept my nose out!

See! I am learning even more because of this.

> you take this as a critic, which is not.

Interesting. What is it?

Well I was about to say correction but I think I got caught by a false cognate so it was indeed a critique but you weren't criticizing the previous post if my understanding of those words is correct.

OTOH, what I see is misuse of words like this ultimately ends with “well, lots of people use it like this, so now it’s correct.”

(See, for example, “literally,” or “comprise,” or “hopefully.”)

All the "incorrect" uses of those words are fine and are a part of the language. Outside of prescriptivism, that's just the use of the language, not misuse.

Even "literally"? While my original comment was just a Stannis joke, I really think you need to rethink how useless it is to have a word that means both "not figuratively" and "figuratively" (by common usage) at the same time. If it means both, we literally have no further use for it in the English language because it modifies nothing.

Why not have the word "no" mean "yes" and "no" while we're at it?

Cleave:

- to adhere firmly and closely or loyally and unwaveringly

- to divide by or as if by a cutting blow : split

https://www.merriam-webster.com/dictionary/cleave

So cleave is it's own opposite. It causes no confusion because context is everything. q.v. homonyms.

Edit: "The word "cleave" has two opposite meanings - either to stick together or to split apart. Are there any other words that do the same thing?" https://www.theguardian.com/notesandqueries/query/0,5753,-13...

    The problem with defending the purity of the English language is that English is about as pure as a cribhouse whore. We don't just borrow words; on occasion, English has pursued other languages down alleyways to beat them unconscious and rifle their pockets for new vocabulary. 

    - James Nicoll

"Literally" has meant figuratively for centuries[0]. It's a language construct known as a contronym[1,2].

Yes, English is not strictly typed, and doesn't conform to a formal spec or mathematical proof. A word can have multiple definitions. A word can have contradictory definitions. The definition of a word can change over time without needing to submit to an approval process. A word can mean something different the next town over. Dialects and creoles and slang exist in wild, flagrant abandon and disregard for your rules. And all of it is valid.

Despite all of this, people manage to be able to comprehend one another, even when using literally to not literally mean literally.

Even the user who used "less" instead of "fewer" upthread that started this. Everyone understood exactly what they meant. The two words mean literally the same thing. But some people insist on maintaining a meaningless formality and insisting upon rules that don't matter, or rather insisting that even in casual conversations, their rules must take precedence over everyone else's.

[0]https://www.thecut.com/2018/01/the-300-year-history-of-using...

[1]https://en.wikipedia.org/wiki/Contronym

[2]https://www.quora.com/Etymology-When-did-the-word-literally-...

You're literally doing what you still think I was doing and are literally not realizing the hypocrisy. The "fewer" line was a Stannis Baratheon reference.

Yes it is correct. “Standard” English took one dialect and attempted it enforce it on hundreds of others. It’s authoritarian and not how language works

Yes but I think a soft resistance is still necessary in a society towards language errors otherwise the language becomes too fluid to be usable and understandable. Where to draw the line between information and rudeness is the difficult part.

You don't need to jail or slap someone in the face for using less instead of fewer but you don't want gynecologist to mean oncologist and cancer to mean gonorrhea from one week to another otherwise nobody knows what we are talking about.

Agreed, but is less vs fewer worth anything, really?

You can always shrug it off, possibly what most readers have done.

An odd thing, for a native English speaker to be given advice on the language by someone who is learning it, no?

In English there is both essential plumbing and pointless ornament, and I wish for that distinction to be recognised because confusing the two is damaging.

I am not the one who corrected the comment. I am only someone who said that this correction had a value and I am not telling you how you have to speak english.

My point was thatin my opinion I did not believe the correction had value. I was categorising it as a pointless ornament rather than something actually useful. But I emphasise, that's just my view.

What exactly did you learn? That some people try to enforce made up rules on native speakers and then feel superior for it?

This is a tech forum for discussing technology. This isn't the place for that, and honestly I'd prefer not to have to sift through a bunch of pedantic grammar nitpicking on here.

There are a lot of ways to correct people. While I agree it's clearly not adding a lot to the conversation, the forum is async so it's not really disrupting much, as the comment goes down the comment list.

I'm a non native speaker and surely make a lot of mistakes, and I would be OK with such a correction

It's extremely pedantic and condescending, which are the only emotions I can feel any more and that's why I posted it.

Both are used so both are correct, unless you're a prescriptivist, in which case a saying about stones and glass houses comes to mind.

Prescriptivist? Nah. Mandarin blow rid nails for happy deranged elf nestle corruptive hand wither. If it seems I stopped making sense, it's because I was briefly testing where your binary labels lead. Apparently, there are things between descriptivist and prescriptivist and both of us fall within those categories.

You're making my point for me, as your gibberish is not in use so is unintelligible.

The person you replied to used perfectly common and comprehensible English, so much so that you knew exactly how to "correct" them.

So the comment was grand and your prescriptivist response adds nothing of value. Hopefully you can see why this is true and reflect on why you felt the need to "correct" it.

>So the comment was grand and your prescriptivist response

There we go. The "response" was prescriptivist. The person posting it (me) and yourself are both somewhere in between the two. Now I think you're starting to get it.

Regardless, the original response was a just a joke referring to Stannis Baratheon (from Game of Thrones) saying it. I think some people read it that way. You didn't.

A bunch.

I personally find it "sort of" funny how nvidia is caught between what they need for gamers and what they need for IA, and how they're trying to pull all the tricks to not have their gamer cards eat up their "AI enthousiast" cards sales.

Absurd pricing, ridiculous vram offering, I'm sure they're trying very hard to find a way to stop AI like SD or LLMs to run on their gamer cards at this point.

It's reached a point where not only has ATI/AMD essentially caught up to them in rasterization, but they're frankly a better offer at every price point against the 4XXX generation for pure gaming, with only DLSS and brand recognition keeping nvidia ahead.

Elsewhere in the comments they make a good case for this card being positioned for Enterprise computing and not for gaming or ML.

We just need some fierce foreign competition. I hope the Chinese will soon catch up and able to produce high performance gpus.

It won't happen.

The equipment from ASML and the know how of TSMC and Samsung is far far out of reach of china now.

Intel can't even seem to get down to 5nm.

I wouldn't underestimate China: https://www.extremetech.com/electronics/huawei-quietly-annou...

Meteor Lake on Intel 4 launched last month.

It seems super weird to say that given who they are and how many times they tried, but Intel is my hope right now. Their arc card where very good for a first salvo, they do great stride in drivers and modern tech (XeSS beating FSR on its first try for exemple), and they're really so scared of behing left behind by arm + cuda card that they just might take it seriously this time instead of killing it when it starts working.

I want to be hopeful, but these corporations will never do anything for the consumer. At their core they think in terms of squeezing the customer as much as they can. As Steve Jobs said, when the sales people run the company don't expect a good product.

A wonderful thing would be a GPU with open source drivers and a standartized API. AMD could do that and it would kill nVidia marketshare in a few years, but they don't do that because they want the server gpu money at a very inflated margin. Maybe Intel will do it but that depends who runs the company. With open source often corporations publish a feture stripped solution and keep the full drivers closed source. Sad reality we live in.

I've got my fingers crossed on them cramming a lot of vRAM in there. They were putting 16gb in the Xeon Phi 7110P when 8GB was acceptable for system RAM in gaming machines. If they dropped a 64GB+ prosumer card, I'd be so happy. Even if it was something like 32GB of super-fast memory and another 64-128 of slightly slower-but-unified memory, I'd be ecstatic. Being able to fine-tune a 175B LLM locally would be incredible.

Stopping SD or LLMs from their consumer cards would be dumb.

If 300M people in the US, let alone 7B in the world, need even a 4060 to run basic business workloads, NVidia is sitting pretty.

The reason is because they want to sell entre/medium business workloads cards at 1/2k€, and gamers cards at 4-700€. Right now, it's burning them on both side, and the only "fix" they found was to give just enough vram on their card to not be useless.

That's why you have an 8 GB 4060 Ti then a 16 GB one then the 4070 Ti with 12 GB ... It's a mess.

Why can't they just focus on improving manufacturing output and satisfy both markets by having enough supply?

Fabs cost a lot of money and take a lot of time to build.

Because then they are committing the mortal sin of capitalism: leaving money on the table.

They artificially segment the market because "pro" users are willing to pay almost 10k for a good card. So why not offer just that?

Because people who do AI are willing to pay several k€ per card, gamers are hitting a wall before 1k€. It's the same cards, only the vram change, and the vram is "cheap" to add, comparatively of the total price.

So now you have gamers who are pissed off at their 4060 Ti or 4070 Ti barely having enough vram for 1440p at modern quality of gaming, and AI enthousiast wondering why buy they "pro" card when a 4090 is not even 2k€.

essentially, they're trying to virtually segment a market that is not segmented, to take advantage of the much higher buying power in one of the segment.

AMD is still lagging on software though. I wish they would get their act together but their drivers are just not anywhere near as stable or compatible as Nvidia's. And then there's Intel, and it makes you appreciate the state of AMDs...

For pure gaming on windows I would disagree, as of the 7xxx cards. And I'm one who was on team ATI for a few generations before moving to nvidia because of terrible drivers.

Only reason I still have a nvidia card is the VR issues. Unless it’s been fixed recently they can’t compete in high end VR.

Is nvlink still relevant to AI?

They claim PCIe4.0 makes it irrelevant, but that doesn't really make sense and it's most likely the case they want to charge a fortune for their high-memory options.

It is if you can pool the memory. It's easier than having to do the split of the models in software (though that's a somewhat solved problem) and from what I know allows higher GPU utilization on both cards when they don't have to wait for information to pass back and forth.

it is barely relevant to big players, but is extremely valuable for small players: distributing your workflow to multiple gpus manually is not that simple thing to do and there are a lot of much more interesting/important problem to solve than shoving your model to a gpu.

Extremely disappointing.

The 4090 wins in every benchmark for 1/3rd the price. Why would anybody buy this card? Is 8 GB more VRAM and lower power consumption really worth that much when the performance is so lackluster?

You know at first I was thinking the same thing, but after having 2x 3090's raise the temperature of my bedroom to 86 degrees in the middle of the night last week fine-tuning an LLM, I could see the draw of 64GB (for 2) with 400 watts total less heat than the 4090's in my work space.

Why do they have this naming… it’s just insane. RTX 5000 Ada… do they just put letters in front or names at the back these days? So confusing and the consumer cards will also be rtx 50xx

In addition, the name "RTX" is really gaming oriented (raytracing), so seems not a very applicable name to workstations.

(Not saying you shouldn't game on a workstation, but a workstation card will be worse at it especially for its price)

Products like TwinMotion have made ray tracing more important in the workstation market.

Nvidia always named their microarchitectures after famous scientists: Ada, Ampere, Turing, Pascal, Maxwell, Kepler, Fermi, Tesla.

I guess what's unique with Ada is that they're using her first name? Though most official sources call it Ada Lovelace in full.

I believe the issue with Lovelace is that you may find less than PG results typing that on a search engine. Hence using Ada primarily on the marketing.

I think the complaint is more with the consumer card being 4xxx but this is 5000 both on the same architecture.

Yeah they have different naming conventions on the workstation cards;

Quadro RTX 4000

RTX A4000

RTX 4000 Ada

Unfortunately they’ve had 3 separate naming conventions in 3 successive generations. Those 4000 series cards are in the same position in the lineup for each generation.

I just went through this with our Dell rep. The generations aren't totally successive, if you count the non-Quadro RTX 4000 series, which is Ada generation but not part of the RTX 4000 Ada series.

Add to it the card variants, and there's a chance that you might still end up with the wrong part if your purchaser isn't careful.

Was about to post something questioning the "forever" part because my memory only starts to link Nvidia generations with scientist somewhere around Kepler. And that's despite having followed GPU tech a lot more in the years before. But according to Wikipedia it goes back to the days of the Riva TNT: the wiki seems undecided about Fahrenheit-ness of earlier generations, but I'd consider that close enough for "forever".

I think they started with hotness and switched to scientists as a more sustainable source for names

https://news.ycombinator.com/item?id=38252900

The "hotness" were also scientists

Yes but they historically do not put the architecture name in the product name like that. They also never use the naming scheme of their consumer graphics cards on their workstation line. They've always had a different system. This card breaks both of those conventions. Workstation cards are supposed to be Quadro. But it looks like they've rebranded the line as "Nvidia RTX". I can only assume that was an intentional move to make their lineup more confusing.

They 100% used (at least) Tesla and Maxwell in press releases/product SKUs.

If its confusing enough, some consumers will pay top dollars for crap

Any explanation for lacking NVLink other than having been purposefully crippled?

NVLink is essential for training large neural networks, which NVidia now earn majority of revenue from. Their sales of more expensive GPUs will be affected if they have NVLink in cheaper GPUs.

The issue is that this has caused severe shortages. The only new card with NVLink is at the very highest end and when trying to get a quote recently, I was told there was a 13 month delay in shipping. But if I don't need NVLink, just a few months.

They keep removing features

They are available at 3-7 day delivery here: https://www.newegg.com/nvidia-900-21001-0000-000/p/N82E16814...

At this pricing level, with this amount of RAM, I suspect a lot of use cases will be with ML and GenAI. Benchmarks for these use cases would have been interesting.

To be honest, the best benchmark you can run is your own training code. Everything else is a guess.

When I tested the A6000 against the H100, there wasn’t that big of a boost from the newer card. Perhaps GPU operations weren’t the bottleneck in that case.

> To be honest, the best benchmark you can run is your own training code. Everything else is a guess.

Yes, but the point of a review with benchmarks is that it is expensive and time-consuming for a customer to acquire the hardware just to run their own benchmarks on.

Stable Diffusion and various LLMs are available pretty easily.

A simple benchmark that this version of stable diffusion/llm was used with these settings and this is how long it took to produce image/we got this many tokens/sec would be a nice comparison that you are in a good position to do with access to all the hardware.

This would end up memory-bound for LLMs (and you would better get x3 4090 for this) or by compute for diffusion models (and again 4090 is like 10% slower than H100 for 1% of the price).

I get what you’re saying, but I think too much complexity hides in those numbers. Some of the optimizations for LLMs (flash attention) are hardware specific.

And anyways, it’s not that expensive. You can rent the same gpu from cloud providers for a few dollars per hour. If you are serious about buying a GPU it is an extremely small cost in terms of time and money compared to the price of the gpu itself.

The alternative is trusting an online review running some training or inference code which is likely not comparable to what most people are doing.

> You can rent the same gpu from cloud providers for a few dollars per hour.

The big three cloud providers don't offer things like the RTX 5000 32GB or the 4090 do they?

AWS will happily rent me a H100, A100, V100, K80, A10G, T4, or M60. Not to mention a Trainium, Inferentia, Inferentia2, Gaudi or Qualcomm AI 100.

And don't forget to benchmark each one of those in a 1, 2, 4, and 8 GPU configuration; and with a variety of batch sizes; and with and without distributed training. Remember to work out the performance-per-dollar for on-demand, reserved, and spot instances, times three different cloud providers. Now, to compare to on-premise pricing we start by calling our air conditioning and backup generator vendors...

I don't see why the parent comment would have meant the big 3 cloud companies. The GPU rental space is pretty commoditised, unlike for instance CDN/object storage where staying locked into a single vendor makes things simpler.

Not explicitly, but RcouF1uZ4gsC talked about "an extremely small cost in terms of time and money"

Starting a VM with a cloud provider you already know and use may be extremely fast, but adopting a new cloud provider isn't.

Even if your organisation is so unbureaucratic you can get a new provider set up without any due diligence work, every cloud provider comes with their own oddities. Will they have a wacky set of default firewall rules? Will they only offer Debian not Ubuntu, if you want an instance with the CUDA drivers already installed? Will they insist you learn what a 'provisioned iops' is before letting you start an instance? Will some GPUs only be available in certain regions? Will accounts have quota limits that vary by region, availability zone and GPU type, some of which default to zero?

This isn’t migrating your entire app to a new ecosystem, this is starting up an empty machine, cloning a repo, opening Jupyter, and running some dummy training tasks with dummy training data. It takes 5 minutes from setup to teardown. It’s not difficult.

FinOps is a full time job - https://www.vitado.com/finops-how-to-gain-sponsorship-and-im...

I don’t want to be promoting any particular cloud GPU services but you literally can rent those GPUs in single and multi GPU configurations if you need to. Even for H100 it’s only around $2/GPU/hour and comsumer grade ones like the 4090 are even cheaper of course.

Content creation tools (Maya, Blender, in-house) are another good candidate. Any game level editor would much apreciate extra GPU memory :)

it is 20-30% slower than 4090 or H100 in compute, the only improvement is slightly more RAM. This card is not for ML (on purpose) - it is for more enterprisish tasks: some advanced video streaming/rendering, virtualization, etc.

It's for some ML tasks. Just not large language models.

If you're making, say, an ML-based on-premise CCTV system and you need to run several large ResNets at the same time? And you don't want to go rack-mounted, as some sites don't have a data centre? And you want the longer lifecycle and guaranteed spare parts availability of an enterprise product line? This could be the card for you.

Admittedly it's a rip-off, but the Workstation/Quadro line always has been.

Honestly I'm not sure how healthy the workstation market is right now - with the rise of work-from-home and hybrid working, I don't see many people using huge desktops any more. And when Adobe puts a powerful generative AI feature into Photoshop, they don't expect users to upgrade to powerful GPUs - they run it in the cloud, so it works for users with puny GPUs and Adobe can get that sweet sweet recurring revenue.

No, this cards would be inferior to 4090, that you can get everywhere and you can buy 3 of them for "spare parts"

Comparison with 4090 is irrelevant as 4090 legally cannot legally be used for commercial usecase, and most companies just follow the law.

Sorry what does that mean? There are many people creating content and software for commercial use cases with these cards and I've never seen a EULA anywhere preventing this.

YetAnotherNick is probably referring to the geforce eula [1] which says:

"Customer may not [...] provide commercial hosting services with the SOFTWARE. [...] The SOFTWARE is not licensed for datacenter deployment, except that blockchain processing in a datacenter is permitted."

So if you were e.g. an ad agency artist using a 4090 at your desk to generate images for commercial use - that's fine, because it's not in a datacenter, and the commercial services you're providing with it aren't hosting.

I've never heard of nvidia taking any enforcement action, or defining precisely when an office becomes a datacenter - I suspect this is mostly to ensure the big cloud providers don't offer 4090s.

[1] https://www.nvidia.com/en-in/drivers/geforce-license/

Yeah I'm aware of that but that's pretty distinct from "cannot legally be used for commercial usecase".

I can't edit my comment now, but by commercial I meant "using 4090 for commercial purpose" that is renting, not not being allowed to create commercial content using the card.

Reading my comment again, obviously it wrong and I should have been more careful with the wording.

This limitation doesn’t really apply anymore with the open kernel driver. They can’t stop you from using a GPL-licensed driver in a datacenter or any other way you want.

Why/how is this even legal?

Wait, they explicitly allowed to use cards for blockchain and prohibited for anything else?

I wonder if places like RunPod have a special agreement with NVidia, there's some way they're working around this (not installing the drivers?), or if they're just flagrantly offering it as they offer several consumer card options in what's presumably a datacenter environment.

Lot of companies offering 4090 leases, heavily doubt that only cloud gamers ans AI hobbyist rent them.

If you have any sources for that claim, you should post them here. As it stands, it's a pretty unusual statement.

Lol, what? Of course it can. Nvidia can say they don't allow it in their user agreement, but last time I checked user agreements don't form actual legal law. Nvidia can sue you for breaking the agreement(I wonder what for, exactly?) But that's about it.

The worst thing that can happen is Nvidia declining any warranty repairs on these cards as they've been used outside of the intended use. But it's definitely not a legal issue.

And EULAs in general are just not enforceable outside of the US.

IMHO it's the against the EULA to rent a 4090 - and this is why the big cloud platforms don't offer any. But using a 4090 in a company's own datacenter should be fine.

RunPod is fairly large and openly advertises the consumer cards. I wonder if they have an agreement?

Weird timing, reviewing it months after it was released.

Reply on news.ycombinator.com