What if the RAM/GPU shortage is deliberate? | Hacker News Reader

What if the RAM/GPU shortage is deliberate?

xn--vk5b17r.online

55 pointstheoneone22 comments

From the opposite angle, there's apparently evidence that the RAM companies themselves are price fixing: https://www.tomshardware.com/tech-industry/samsung-sk-hynix-...

The existence of a lawsuit is not evidence.

That's false. It's evidence, but it isn't conclusive. Someone thought that the law suit was worth filing, and that could be because the law suit has a chance of succeeding. It could also be some other reason, and it could also fail even if completely good faith -- this is why the evidence is non-conclusive.

By your logic, counter-lawsuits and especially the SLAPP suits are also evidence.

Still, RAM manufacturers have a long history of price fixing.

https://en.wikipedia.org/wiki/DRAM_industry_price_fixing

> Someone thought that the law suit was worth filing

No, someone thought that they could win a lawsuit.

We live in a highly litigious society. This happens all the time.

It's not _evidence_, it's _suspicion_, and it's asking it to be reviewed in a court of law.

The memory manufacturers don't believe the rate is sustainable, so are hesitant to start up new production to meet demand when it will inevitably crash. They have to operate on longer timescales than software companies.

I think this misunderstands basic supply and demand.

If there's this much global demand for RAM, there is simply no need to speculate further about intent; all else being equal, prices would have risen naturally anyways.

Of course, the rise in prices is also incidentally convenient for those that have RAM, and would make it harder for others to compete, but whether this was intentional is beside the point.

A single bit of HBM consumes 2-3x the manufacturing capacity that a single bit of conventional DRAM does.

So you're arguing that ai companies overbought memory to increase scarcity? Possibly even buying e.g. consumer ddr?

just a theory! The end result was too make it difficult for me and you to run llms on our systems, and you need porwerfull system to run them. But some months ago, those powerfull systems where affordable for end consumers. Now, good luck buying let's say 128gb ddr5.

Do you believe this is more likely than corporations trying to restrict the supply for their corporate competitors? My thinking would be that we're just caught in the crossfire. The percentage of people seriously running their own models locally would hardly be any less diminishing if hardware prices were at their previous levels.

absolutely!

Hyperscalers and social media companies hired tech talent to keep folks from starting their own companies and keep other companies from hiring them.

memory companies are known to price fix as well, so i could really see either possibility tbh

and why not both?

This has been suggested several times in the past by others: Data center builders bought up all RAM production for the foreseeable future to force anyone wanting any serious compute to become a customer of said data center builders. Unclear whether it's deliberate or just a convenient side-effect for said data center builders.

You are aware that a 1 trillion parameter model won’t run on any quantity of consumer hardware, even heavily quantized… you need a data center for that. And that is not even to talk of the Kimi K3 model which is in the ballpark of 3 trillion

You do not need a data center - just a single rack will do to run it fast. Or about 100x 5090s.

You don’t need a datacenter. 1T model requires 1T of RAM. Not easy to come by today in a compact factor but nowhere near the datacenter.

> 1T model requires 1T of RAM

assuming 8 bits per weight, but 4bits is commonly used once you get to that scale.

Its certainly plausible but what % of consumers would actually buy RAM and do it themselves? The most profitable regular mainstream user would gladly pay $20 and use a simple web interface and not worry about anything else.

Some personal thoughts on the current ram/GPU shortage -price hikes.

For this to be true, and pointed at consumers, it would have to be the case that consumers are a threat to frontier models. To be a threat to frontier models consumers would have to either 1) be willing to run massive hardware or 2) accept results that are no where near frontier levels.

On point 2: Everyone has different standards and goals but after testing a lot of local models on different workloads, I wouldn’t say the results are acceptable. I say that using 4 local models for different things on a daily basis, but to get to that point took weeks of testing and tinkering to get the quality to an acceptable level (for each one!). Consumers aren’t going to do that. Maybe someone who sinks 40k into GPUs will, but that’s not representative of consumers (or most developers).

Which brings me to point one; the number of people who are going to rack an Epyc or Xenon system so they can run 2+TB of ram or run extension cords to different circuits so they can run more than 4 GPUs is so tiny they simply aren’t worth caring about (are you getting ready to argue about power draw and circuit capacity in different countries? You are one of very very few).

Did the AI companies lock up future production to keep their _competitors_ from getting more memory? Obviously yes. But that’s not a conspiracy, it’s just business.

I don’t mean to be dismissive. I think you are directionally right. Local AI will eventually displace the frontiers for all but the enterprise. I just can’t imagine the meeting about capex includes the thoughtful, “let’s add a couple more zeros to keep Jane & Jim developer from running Kimi at home until 2030.”

Reply on news.ycombinator.com