Hacker News Reader

Comment by thephyber | Hacker News Reader

I don’t think we should use current prices as landmarks for large scale demand. That Studio’s current prices is inflated because of a (presumably) short term supply crunch, not because the average user is willing to pay $24k for a home AI inference device.

It assumes that RAM remains supply constrained and that none of the existing RAM contracts are cut short.

But Meta and xAI putting A TON of AI compute onto the market. OpenAI and Anthropic are raising the costs of inference (by reducing how much inference users get via subscriptions). And we haven’t seen Oracle / CoreWeave struggle to pay their debts yet, but they will be selling assets once they get close to that point.

View on HN

512GB M3 Ultra is out of stock, not coming back, and there’s nothing like it on the consumer market. That’s the reason they go for so much.

But there will be and his point is they cut off supply to make room for the new M5 ultra which I hope has 768GB or more of memory.

Let’s see, I hope!

By the way, they cut the supply because they run out of RAM. They still sell the 96 GB model. Everything else is gone, no stock, they cannot manufacture them. Even minis are 48GB max!

Yea but how many of them are selling at that price? Dozens?

Funnily the current high end Mac Studio are not suited for current LLMs. M3 Ultra is "quite an old" chip for AI, despite its bandwidth. The issue for running local models (especially LLMs), you need few things to align really well: 1) compute power (affecting PP) 2) VRAM capacity (affecting model size you can load) 3) Bandwidth (somewhat affecting decoding speed).

The issue with the M3 chip is the compute performance, as it doesn't fit well the transformer architecture. This changed with the M5 (apple baked their own matmul into the chip), which would significantly speed up PP (and video/image generation btw), making the M5 Ultra significantly faster than the M3 Ultra and in practice much more usable. You can try to load Kimi or GLM on M3 Ultra, but it's not usable. Now the M5 Ultra is not out yet, but undoubtedly it will be a superior offering, and shilling 15k on 512GB version is actually reasonable (if it's every priced remotely around that tag).

Do you honestly think that’s families buying them?

My money is on small companies or affluent programmers experimenting with some new hobby / business model.

What makes you think the supply crunch is short-term?

Unless the raw materials have an inherent limit on mining/production due to the amount present on the planet, why should or would companies not ramp up to eventually meet demand?

Edit: Okay, this doesn’t mean that that’s actually possible in the short-term, so I think you’re right. But that means as the silver lining, in the medium term horizon there’ll be enough supply again? :’)

> why should or would companies not ramp up to eventually meet demand?

Memory is a cyclical market that has historically rewarded conservatism [1].

Counterpoint: there is enough demand from enough capital-rich customers that they may be willing to shoulder the capital risk.

[1] https://www.ldeepai.com/tech-hub/dram-industry-consolidation... Sorry for the slop link, it has a good chart from a solid source

Thanks for the link (and underlying thoughts), I really hadn’t considered that.

So essentially, due to technological progress and other factors inducing price collapses (or at least cycles), you can’t start stockpiling insane amounts of finished-product semiconductor, which means you can’t scale production at current technology levels to infinity either?

I heard that China was spinning up DDR5 (but not HBM?) production in the next couple of years, with the hope of outcompeting Korea and Taiwan in the mid to long term.

It does seem like an opportunity on a silver platter for Chinese newcomers. Huge demand at the moment.

In two to four years, the Chinese will have at least half the memory market worldwide, and once in, they will continue forward and not look back.

I also believe there would be one or two tech companies that will get into memory by taking it in-house to make sure that they won’t have this problem again in the future.

Don't forget price fixing [1] which we are seeing clear indicators of happening right now too.

[1] https://en.wikipedia.org/wiki/DRAM_price_fixing_scandal

For existing producers expanding capacity would be a risky move. But it's the perfect time for any newcomers to enter the market. Low yields and worse product don't matter as much right now, and by the time the market cools down you have everything dialed in and can compete on even ground

> it's the perfect time for any newcomers to enter the market

This is a good hypothesis. Curious if anyone has data on the failure rates of new entrants in semiconductors based on how frothy it was on founding.

On one hand, more demand makes selling easier. On the other hand, a shortage makes your input costs (consumable and capital) pricier.

EDIT: It seems like the 2 to 3 year lead time and a crowding effect from new entrants historically made booting up a fab into a boom a bad bet [1]. (The article argues, convincingly, that this time may be different.)

[1] https://www.uncoveralpha.com/p/every-memory-cycle-ends-the-s...

If demand doesn't fall down or current manufacturers supply go up, somebody (presumably in China) will spin up fabs. Apple wanted to use blacklisted Chinese RAM already.

I am hopeful but I am not confident China has the capability to do it.

They're already doing it

https://www.techspot.com/news/112502-memory-prices-tipped-fa...

As if it requires some unique genes or brains?

If a place can do it, another place, with a huge track record on manufacturing and lately expanding all kinds of tech, can.

It might help to not have much in the way of environmental, safety or labor regulations.

Whether or not you feel like those are good overall (I do), they do actually also slow things down.

>It might help to not have much in the way of environmental, safety or labor regulations.

Yes, like how it helped western industry early on. Or, well into the 70s for the most part.

where is this cheap Chinese RAM? I'd like to buy some

Ebay and Amazon are flooded with it. Especially if you are looking for anything prior to DDR5. DDR2 and DDR3 are especially flooded with weird brands you've never heard of before.

Unfortunately its not so cheap anymore as everyone ramped prices up of course.

Last year I could still get 32GB of DDR4 for under $60 from chinese brands.

DDR3 is from 10 years ago. DDR2 is from 15.

And?

I just upgraded my 2008 Thinkpad R61i to 8GB of DDR2 a few months ago while I was also upgrading to a core2duo.

DDR2 and DDR3 are still in active use by SBC manufacturers.

> I just upgraded my 2008 Thinkpad R61i to 8GB of DDR2 a few months ago while I was also upgrading to a core2duo

Uhhh… ok… good for you I guess…

Some of us keep the worthwhile trash alive forever. This laptop is gonna be twenty years old soon and it's too comfortable of a machine to be tossed to ewaste.

It's the euv machines that are the bottleneck. Pretty hard to ramp those up any faster in the next few years.

DDR5 is still mostly made with DUV (remember Intel 14+++++++++?), and even though manufacturers have slowly been moving a few layers to EUV the advantage is at the margin. Lack of EUV at scale will not prevent China from ramping useful RAM into this market.

spinning up fabs takes ages, micron has a US fab started bulding in 2023, its still not operational (projected to start mid-2027)

That proves that spinning up US fabs takes ages.

Chinese fabs might not be so tied with red tape and regulation upon regulation (which is a funny reversal, in terms of "communism vs capitalism" bureucracy/inefficiency cold war thinking)

The Chinese don't have access to new EUV machines.

All of their fabrication ability is based on old processes.

Not so sure about that: https://www.reuters.com/world/china/how-china-built-its-manh...

DDR5 is DUV not EUV

1.) China is not communist, even remotely so. China is fascist in every sense of the word.

2.) Authoritarianism can move faster than anything. They can just say "wipe out that village, build the coal plant there, data center here, fab here.

3.) If it's red tape and regulation holding the US back, then that's clearly not "capitalism."

>1.) China is not communist, even remotely so. China is fascist in every sense of the word.

Except in the actual historical sense. They appear to enjoy all sorts of freedoms, increased prosperity, even have elections at different levels but under a single party system. Which is not necessarily that different than a effectively two party system.

>2.) Authoritarianism can move faster than anything. They can just say "wipe out that village, build the coal plant there, data center here, fab here.

Now that China is more effective, "it's easy because they're authoritarian". Before the argument was "authoritarianism can never be as effective as free-market democracy".

>3.) If it's red tape and regulation holding the US back, then that's clearly not "capitalism."

It's real world capitalism, not some fantasy some guy imagined removing all warts.

I think the most ironic fact of the 21st century is that there are less than 20,000 naturalized citizens in China. Western leftists don't really have a good explanation for that one and it definitely leans into the fascist characterization.

How is that ironic or necessary? Why should they hand out citizenships, when work or resident permits will do?

And that's somehow fascist, as if the problem with fascist Italy or nazi Germany was that they didn't hand out citizenships?

If the increased demand is not short term, production capacity will eventually increase. In the meantime, the logistics disruptions and industrial material shortages and energy inflation will disappear as soon as the wars disrupting them stop, which should bring prices down.

you assume demand will remain flat

what if demand keeps rising faster than production capacity is deployed?

If demand and prices keep rising without production capacity being built fast enough, there will likely eventually be a rush leading to overinvestment and price crashes, but there are too many other factors involved; state investment for security, international politics and trade relations, the possibility of an AI bubble burst, etc.

There are wars coming. The prices are not going down.

We are in a bubble which will be burst the moment the world starts retaliating against the US' 20+ year history of supporting genocide and committing war crimes unabated.

Buy the AI toys while you still can.

DRAM doesn't require cutting edge Fabs, and Chinese manufacturers have finally caught up to the DDR5 standard.

> Multiple motherboard and PC component makers move forward with Chinese-made memory validation

https://www.pcgamer.com/hardware/memory/multiple-motherboard...

The only thing standing in the way of a major Chinese DDR5 ramp up is money.

The underlying demand for massive RAM builds is coming from AI hyperscalers.

There are lots of signals that the sector has been overinvested and that corporate customers are pulling back on spending as the cost of the APIs is revealed.

Once the hyperscalers start struggling to bay their debts (it will happen, just a question of time), there will be a supply glut.

So the only question is: do we share the same definition of “short term”.

Reply on news.ycombinator.com