Hacker News Reader

Comment by ilaksh | Hacker News Reader

Cerebras already runs large models like Kimi 2.6 or GLM at like 30x speed. 100 times is next year, not six years.

You can actually test it out on their website, just imagine 3 x faster and maybe 15% smarter.

Cerebras is literally the entire wafer, so it can't get bigger. So where is the jump from 30x to 100x coming from? Node improvements only yield like 10-20% gains these days...

They have a next generation, I don't really know if it will be 3 x or what but I heard it was significantly better.

Also there are other people innovating in hardware.

Could we not just make bigger wafers, if the technology called for it?

The investment in bigger machines at the fab might set you back billions. I don't know about the lithography technology either, how easy you can scale it to larger wafers?

we seem to be in a phase of spending trillions on the computer, so while it isn’t likely, it isn’t impossible

There's so much you'd need to scale, it's even more expensive than the humble few billions you spend on a new fab. And, beyond R&D, the scale itself also makes it significantly more expensive. This is an energetic dead end, you would need a country to fully commit to making this one thing.

Lithography is AFAIK the most expensive part. If you wanna scale to larger wafers I'd guess you're talking hundreds of billions and at least a decade. And even then, what will your yield be? Surface area grows to the square of the diameter of a circle, and so will errors. The larger your wafer goes the lower your yield will go, especially if you're going to use the whole wafer in one final product.

You also need to worry about yields, Apple, AMD etc can sell ”bad” chips as lower core versions, if you’re depending on whole wafer you have little room for error.

Well they still just do the same thing GPUs do where bad clusters are disabled. There's going to be a balance there for how big the clusters are and how many tiers they'll offer, but that's a known problem which can be well modeled.

Love the idea of discounts based on model error. ”this one doesn’t know what butterflies are, it’s on sale for 8% off”

Yes, "this one is obsessed with the golden gate bridge" will be for real this time.

Cerebras does the exact same thing, disabling the bad tiny cores on their wafer-scale and routing around them in their compute network. The question is how much extra capacity do they have to plan for, and does the cost of that still make sense, with the yield they can get.

Cerebras pretty much has to be at the end of what can fit on a single wafer. Larger wafers would require retooling one of the most up-front-expensive industries, and denser is not arriving fast enough. I expect their scale-out story to rhyme with NVidia et al working at rack scale and beyond, just denser. A rack of Cerebras has 400 G networking for two wafers today.

Larger wafers has been attempted for over a decade now and doesn't seem to be going anywhere. See the 450mm wafer section of https://en.wikipedia.org/wiki/Wafer_(electronics)

Reply on news.ycombinator.com