Hacker News Reader

Comment by vb-8448 | Hacker News Reader

Am I the only that thinks that anything similar to AGI will come not from raw model capacity but from model speed and efficiency?

In my experience the harness is more important than the model, and anything able to run at 700tps will be the "next big thing".

PS: assuming the current architecture is the right one

View on HN

Why would speed matter? Surely an AGI could think slowly but still be an AGI

Imagine Luna at 10x tps and 1/100 of current cost.

At that point you will be able to "brute force" basically everything.

IMO also a lot of problems with memory and context rot will be solved too.

You can imagine with more operations being available to be done more cheaply and quickly the LLM doesn't need to "one shot" a solution. It could try many solutions, test them, throw some away, wiggle some of the parameters like a genetic algorithm, see how that changes the result, and converge on an optimal solution (based on whatever the cost function is). Basically producing a good result could become like an optimization problem. That would be way too expensive and slow right now.

Currently AGI has been re-defined mostly as "can do any tasks (of the same modality) a human can, equal or better". I feel the "equal or better" includes properties like how fast they can get the task done, not just that eventually they can succeed at it. If you want to have AGI stock buy/sell for example, speed of decision will matter, and there are many other use-cases for which it would matter, not all, but definitely for many tasks, speed does matter.

I think because we have handicapped them with a set of tokens from human language. But (at least I think so) thought happens outside of language.

So more tokens/variability and slow or fewer tokens and fast.

There seems to be a threshold tho, like taalas is super fast but that model is so dumb, being dumb faster doesn't work, seems to be some minimum requirements.

Reply on news.ycombinator.com