Hacker News Reader

Hacker News

Comment by dwighttk | Hacker News Reader

Parent Full thread

dwighttk

Maybe .1% of tasks need this RAM, why are they charging so much?

View on HN

I don't need 512GB of RAM but the moment I do I'm certain I'll have bigger things to worry about than a $10K price tag.

This is Pascal's wager written in terms of ... RAM. The original didn't make sense and neither does this iteration.

I would still wait until I need it before buying it…

Because the minority that needs that much RAM can't work without it.

In the media composing world they use huge orchestral templates with hundreds and hundreds of tracks with millions of samples loaded into memory.

Because the .1% is who will buy it? I mean, yeah, supply and demand. High demand in a niche with no supply currently means large margins.

I don't think anyone commercially offers nearly this much unified memory or NPU/GPUs with anything near 512GB of memory.

Maybe because .1% of tasks need this RAM, it attracts a .1% price tag

With all things semiconductor, low volume = higher cost (and margin).

The people who need the crazy resource can tie it to some need that costs more. You’d spend like $10k running a machine with similar capabilities in AWS in a month.

It enables the use of giant AI models on a personal computer. Might not run too fast though. But at least it's possible at all.

What is stopping us from running these models on a PC with 512GB RAM?

You have a point; technically they aren't impossible to run if you have enough system RAM (or hell, SSD/HDD space for that mater). But in practice neither running on the CPU, nor on the GPU by constantly paging data in and out of VRAM, is a very attractive option (~10x slowdown at least).

So the only reason the mac is faster is because the RAM is accessible by its GPU, right? Not because the RAM is faster than regular RAM, because AFAIK it isn't far off from workstation RAM speeds.

The RAM is faster. 8 DDR5 64GB sticks at 8800 MT/s would in theory give you a maximum of 563.2 GB/s. Whereas the M3 Ultra is 819 GB/s.

Can anyone even get 8 sticks to run together at 8800?

I'd say "No". IIRC the current number is somewhere 5-6K

The narrower the niche, the more you can charge.

I think the answer is because they can ( there is a market for it ). The benefit to a crazy person like me that with this addition, I might be able to grab 128gb version at a lower price.

because they know there will be a large amount of people that don't know this much ram but they'll buy it anyway.

because that's how much its worth

Its not though. For consumer computers somewhere 1k-4k there's nothing better. But for the price of 512gb of RAM you could buy that + a crazy CPU + 2x 5090s by building your own. The market fit is "needs power; needs/wants macOS; has no budget" which is so incredibly niche. But in terms of raw compute output there's absolutely no chance this is providing bang for buck

Do you understand that it's UNIFIED RAM, so it doubles as vRAM? I would love to know what computer you can build for <10k with 0.5TB of VRAM.

2x 5090s would only give you 64GB of memory to work with re:LLM workloads, which is what people are talking about in this thread. The 512GB of system RAM you’re referring to would not be useful in this context. Apple’s unified memory architecture is the part you’re missing.

How much VRAM do you get on those 2x 5090s?

How much would it cost to get up to 512gb?

Reply on news.ycombinator.com