Hacker News Reader

Comment by brotchie | Hacker News Reader

There’s still 50-500x cost reduction in “this is only an engineering problem” low hanging fruit from specialized chips to run the models + improved distillation.

Entirely feasible that by 2031, Fable 5 (or greater) intelligence level models will run cool on smart phones, if not sooner.

View on HN

I saw a 1B model yesterday that was fine tuned on Fable output. I found that hilarious, but it did actually make all the scores go up.

(Actually talking to it, it was about as coherent as you'd expect, i.e. 3/10)

The floor for "actually usable model" keeps dropping though. (Seems to be about 27B right now?)

You're betting on getting getting ridiculously powerful chips to run on batteries in a tiny housing without cooling, while we can't even get enough RAM? It would be a terrible waste of resources. Now we already have TFLOPs wasting in our pockets and backpacks, then we'll have PFLOPs idling, because there's so much time between prompts. Much more efficient to batch it on a server.

At some point we'll have enough RAM, surely. The incentives to produce more are huge and all the fabs are booked out.

Maybe it'll take 10 years or 20 years. <5 years is not long enough for manufacturing to catch up.

Not much of a comment on the phone stuff but I'd caution against suggesting technology will never be good enough to do X. Maybe it'll be horrendously wasteful but it might happen.

If fable 5 could eventually run on a smartphone, I wonder what we'll get out of datacenters. Musk and others are trying to build 100gw of compute by 2030. Will we get something 1-10 million times better than Fable 5, or will the parity gap between local and datacenter capability pair down enormously?

Reply on news.ycombinator.com