Hacker News Reader

Comment by linzhangrun | Hacker News Reader

Thinking that five or six years from now, Fable-level intelligence could be provided at 100x the current speed... makes me feel lost. I cannot imagine what the future will look like.

View on HN

It's not just speed, it will consume a lot less energy per token, maybe even more than 100x difference. And cost for a chip that runs that one model will also go down a lot once volume scales up. They will end up way cheaper than flexible GPU chips.

I expect AI models chopped up into building blocks where 99.9% of the compute is fixed but glued together with flexible "fine tuning" layers that will adapt them to specific applications. Those kind of chips will run 99% of consumer AI and at some point be integrated into consumer devices.

Ohhh that's a very interesting hypothesis. I think you might be right.

It tells me that they have some kind of insider knowledge that the models have hit their limits and won't be getting much better, and it makes sense economically speaking to just bake the current models and use them for the next 5-10 years. Looks like we're near the top of the S curve.

What would possibly tell you that?

Expensive as fuck to make chips, only makes sense if you believe whatever model you're creating a chip out of will not become completely irrelevant in 5-10 years.

There’s lots of use cases where the current models do fine though. A chip that can run a current cheap model at 100x would be amazing for things like detecting prohibited content on Facebook. You don’t need the 2030 equivalent of Fable for that, you need something that can cheaply process insane numbers of posts per day.

No, it makes sense if whatever model you're creating a chip out of will not be irrelevant in 5 years even if it is much much faster & cheaper.

There are probably limited applications but not zero.

That’s not what we are empirically seeing though. There is also no way of knowing that things can’t be progressing for the next 5 years. No foundational model company would be spending the amount they are on research if they thought it wasn’t going to pan out

A model you can run for practically no cost is a new proposition. It is the CPUification of AI. Sure there be supercomputers but you PC will be pretty super too.

I’m not sure we are at the top. Swarmed lower tier models appear to scale up in competence. So something like a Fable swarm running at 100x the speed might provide surprises.

But ”top of the curve reached” feels like the likelier scenario.

No knowledge (insider or otherwise) is needed — could just be a bet, hedged against all the other bets a company AMD's size makes.

Cerebras already runs large models like Kimi 2.6 or GLM at like 30x speed. 100 times is next year, not six years.

You can actually test it out on their website, just imagine 3 x faster and maybe 15% smarter.

Cerebras is literally the entire wafer, so it can't get bigger. So where is the jump from 30x to 100x coming from? Node improvements only yield like 10-20% gains these days...

They have a next generation, I don't really know if it will be 3 x or what but I heard it was significantly better.

Also there are other people innovating in hardware.

Could we not just make bigger wafers, if the technology called for it?

The investment in bigger machines at the fab might set you back billions. I don't know about the lithography technology either, how easy you can scale it to larger wafers?

we seem to be in a phase of spending trillions on the computer, so while it isn’t likely, it isn’t impossible

There's so much you'd need to scale, it's even more expensive than the humble few billions you spend on a new fab. And, beyond R&D, the scale itself also makes it significantly more expensive. This is an energetic dead end, you would need a country to fully commit to making this one thing.

Lithography is AFAIK the most expensive part. If you wanna scale to larger wafers I'd guess you're talking hundreds of billions and at least a decade. And even then, what will your yield be? Surface area grows to the square of the diameter of a circle, and so will errors. The larger your wafer goes the lower your yield will go, especially if you're going to use the whole wafer in one final product.

You also need to worry about yields, Apple, AMD etc can sell ”bad” chips as lower core versions, if you’re depending on whole wafer you have little room for error.

Well they still just do the same thing GPUs do where bad clusters are disabled. There's going to be a balance there for how big the clusters are and how many tiers they'll offer, but that's a known problem which can be well modeled.

Love the idea of discounts based on model error. ”this one doesn’t know what butterflies are, it’s on sale for 8% off”

Yes, "this one is obsessed with the golden gate bridge" will be for real this time.

Cerebras does the exact same thing, disabling the bad tiny cores on their wafer-scale and routing around them in their compute network. The question is how much extra capacity do they have to plan for, and does the cost of that still make sense, with the yield they can get.

Cerebras pretty much has to be at the end of what can fit on a single wafer. Larger wafers would require retooling one of the most up-front-expensive industries, and denser is not arriving fast enough. I expect their scale-out story to rhyme with NVidia et al working at rack scale and beyond, just denser. A rack of Cerebras has 400 G networking for two wafers today.

Larger wafers has been attempted for over a decade now and doesn't seem to be going anywhere. See the 450mm wafer section of https://en.wikipedia.org/wiki/Wafer_(electronics)

It makes me dizzy. I have no idea what is going to happen within even a year from now, can barely even imagine it.

I'm trying to get the most out of it by redlining my AI subscriptions. Hopefully I'll manage to start a business in my niche. I don't even know if my niche will exist in the future.

Not just you. Hassabis in a recent interview looked a bit nervous when talking about AGI is rather sooner than many expected.

Last month: agents spend 4 days on a hack, humans spend 3 weeks (so far) digging through the slop to figure out what happened

Next time, one of those number will be smaller, and the other will likely be bigger. How long before the analysis side gets too overwhelming to bother with? Probably less than 6 years.

Feels both unreal and dystopian. The speed at which these models are developing is very scary

It will be cool but also violent and terrible.

So, like the present

With more wealth concentrates at the top, yes.

Presumably wealth would concentrate upwards even if AI was never made.

Yes it's a function of the monetary system. Absurd amounts of debt only certain people can access.

So you're saying we need to abolish to monetary system?

Like some kind of pimple, which we can pop from all sides. And then we'll build something different. Something that works.

Reply on news.ycombinator.com