Hacker News Reader

Comment by Beach_Birds | Hacker News Reader

What's keeping this from becoming AGI? Are there any remaining barriers? I have to admit, I'm a bit panicked.

Like jerpint said, there's lots of limitations here still since this only addresses the high level decision making part, not the low-level control (sending signals to motors). Not tackling low-level control means the robot is reliant on human-engineered primitives such as 'place' or 'push', which can do a lot but is not fully generic (there are tasks that require dexterous control, tight visuomotor control loops, etc.). In other words, this kind of system is not even close to being general enough to drive a car.

Also, this has the typical limitations of LLMs - no (long term) memory, inherently limited context window size, no online learning via eg trial and error, no 'agency' in general (it can only do things when it is told to, there is no sense of exploring the world and learning about it). To really become AGI these sorts of systems would really need to be able to adapt to lots of niche tasks that they don't apriori know how to do, and there's not a great solution for that right now (prompting can sort of work, but again memory/context window size is still an issue).

Instead of putting those missing functionalities into LLMs, I think those can be distinct components. Nonetheless LLMs undoubtedly plays a huge part in this setup. Much like how the linux kernel was the missing key component of the GNU system.

>> What's keeping this from becoming AGI?

What's keeping it from becoming A G I is that it's still missing a "G" and still missing an "I".

>> I have to admit, I'm a bit panicked.

Try to think of the article above as an academic demo. That's similar to a tech demo, but it only works in a lab, rather than only working in a showroom.

But it's a cool demo, with a cool robot doing cool stuff. Don't let me sound like I'm dismissing the obvious impact of it.

Nothing is keeping this from becoming AGI (for some definition of AGI), and you should not be panicking, remember that nobody can predict the future, especially those who claim they can, and that includes yourself.

Well, I am sure you can come up with some definition of AGI that already fits the kind of models we have now. But let's stick to a definition that resembles what most people would agree upon about what AGI is, as vague a definition as this might be.

And "nothing is keeping this from becoming" is quite a confident prediction of the future, which you said nobody could make. Well, maybe not, but how can you be so sure? There might be fundamental issues with current ML that you just can't scale away.

Remember mechanical computers? They were improved upon for decades, but there was no chance at all they would ever reach the completeness and computing power of modern electronic computers. A new paradigm was needed for that, transistors had to be invented.

The same might apply to current approach to "AI" through ML. Maybe, maybe not. So let me pull an old quote for all of you who claim current ML can evolve into AGI: Show me the code. Show me the program capable of running AGI on hardware which is basically still just flipping transistors. Then I believe you.

"Battle not with monsters, lest ye become a monster, and if you gaze into the abyss, the abyss gazes also into you."

There is nothing scarier than to think that next word predictor can be sentient, because it means we are just next word predictors, and all our pain and suffering is beyond meaningless.

In the same time I think there are definitions for AGI that can fuel exponential development in all important fields (from protein folding to medicine to enegry and etc), and this can just as well be a transformer large enough that has reasoning.

Sorry, but you have a logical fallacy there (affirming the consequent). It does not follow, that humans and LLMs have all same properties, even if they have one same property. ("sentient being is text predictor" does not follow from "text predictor is sentient being").

that is true, I meant that if sentient being is a text predictor then it is possible to build sentience with a text predictor

Ok, so now that sentience is at the table, let me ask you again: show me the sentient code. And a program that takes a token sequence as input, does a single pass to produce an output token sequence, of which it does not hold any state, requiring you to append the output to the previous input to create the next token sequence to feed into the model, for the next single pass output sequence, rinse and repeat, is NOT going to do the job of becoming sentient, no matter how impressed you are by a natural language representation of the token sequence.

Btw I an not scared, just unconvinced. And you arguing like a high priest with all that talk about pain and suffering ain't going to change that.

I could do it with genetic algorithms and scalage.

Reply on news.ycombinator.com