Hacker News Reader

Comment by whimsicalism | Hacker News Reader

There are just so many unsolved problems.

Taking sequences of actions, having a general model of the world beyond just language, memory, etc. are all large ones off the top of my head.

View on HN

> Taking sequences of actions, having a general model of the world beyond just language, memory, etc. are all large ones off the top of my head.

Google: "Robots ground large language models in reality by acting as their eyes and hands while LLMs help robots execute long, abstract language instructions"

https://say-can.github.io/

Released today. An example:

Given the task "I spilled my coke, can you bring me something to clean it up?", SayCan successfully planned and executed the following steps 1. Find a sponge 2. Pick up the sponge 3. Bring it to you 4.

Sure, there are problems that remain. But how can we say there are "so many" of them? We've seen large leaps in capabilities from architectures based on transformers. We've yet to see the limit of transformers as the scaling curves haven't plateaued. It seems to me that we don't know if we're two innovations or two thousand away from AGI. What we do know is that progress has been faster than anyone would have predicted 10 years ago.

Look, I'm actually a massive optimist on this topic and think we can get very far with large language models. Memory I think will actually not be that difficult to solve, optimizing for long series of actions will be a hard one though, as will making an AI that can reason in a representationally invariant and transferrable way (ie. something that can reason just as well about data in the form of images as text).

I'm not saying we're not seeing continuous improvements on performance on benchmarks from scaling up - we absolutely are. But "the end of the benchmarks" is not AGI - even if we have a tool AI/oracle AI that can perform at a superhuman level on benchmark tasks (which I think is absolutely doable within the current paradigm), we do not have a general AI.

Hooking up transformers into an agent AI does not seem like a daunting task given the ability to generate snippets of reasoning on demand.

The thing we’re progressing towards isn’t AGI, if by that you mean is conscious and sentient.

I’m not sure this is actually possible, ie I don’t think we know “general intelligence” exists. The existence of your mind is a trick your brain plays on itself after all. You don’t exist independently of your body.

AGI is AI that is as good as humans on all skills. It has nothing to do with consciousness or sentience.

I think reaching AGI has to do with the 4 E's: embodied, enacted, embedded and extended. These models have no embodiment, no complex external environment, and there is no goal to attain other than language games.

But they could be embodied, and some experiments have shown for example how a LM can guide an agent in a 3D virtual home to accomplish tasks. (*) Maybe AIs can be educated like children after they reach a certain threshold, by giving them robotic bodies and immersing them in the human society which is the most complex environment for intelligence.

(*) Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents - https://arxiv.org/pdf/2201.07207.pdf

They have a complex external environment just as much as we do: the physical world generates their inputs. Just because their inputs are prefiltered through human language doesn't mean they're not exposed to its full complexity just as much as we are, after having it prefiltered through our eyeball and optic nerve. And predicting human statements implicitly requires modeling agents with goals - see the joke explanations.

But LM's can't do causal interventions to separate correlation from causation unless they have access to the real environment. Would a scientist that can't run experiments be able to advance the field by just reading past papers? Even babies need to try causality laws by letting objects fall and observing what happens, we learn by interacting with our environment.

Right, they can't create new evidence to learn from (yet). But we're feeding them a lot of data. Possibly more (and more diverse) data than a human takes in during childhood! I remember a lot of visual input, but I don't remember reading the entire internet. (Yes I know it's exaggeration, but come on- compare characters vs characters).

You could say that predicting the next token is their only action, and minimizing the loss their only goal. It's a kind of impoverished environment. Imagine a human living in a cave, seeing only a scrolling line of text. Would that human learn anything new by getting out of its cave? Plato was so right, intelligence needs access to the full environment.

I mean, isn't the whole point of Plato that we do everything we do, which is a lot, certainly enough to be dangerous, just by sitting in a cave and manipulating levers while watching a shadow? This does not make text-trained AI seem safer.

Seen another way, the lesson of Plato is "Shadows On A Cave Wall Are All You Need - arxiv.com"

Human eyes aren't an input. The process of vision is bidirectional and relies on you choosing what to look at just as much as what happens to be sending photons your way.

Same for the other senses and learning; it's all active engagement with an environment. An ML language model just gets text dumped into its weights until it works though. It doesn't get to ask for more, and it's probably not the best design that we're trying to store everything in its weights instead of letting it look up an external library.

I readily agree that it's not the best design, and that it will have a hard time figuring out things that are not in its corpus. But reinforcement learning is also unlike human learning, and may possibly be better at fitting data into a world model without requiring reflective attention and prediction the way human learning does.

Here's an example of an extended language model operating in a real world environment:

https://www.youtube.com/watch?v=ysFav0b472w

Being good at arbitrarily poorly-designed skills has nothing to do with sentience?

If we can make something equivalent to a human, that is AGI. It does not presuppose some notion of consciousness that we ourselves might not meet.

Nobody actually wants that. You can make something equivalent to a human with your spouse right now and it doesn't solve that many problems.

People want AGI because they think it'll have properties of computer programs like being able to copy it everywhere and have it predictably do what you tell it. I suspect there's an inverse relationship between AGI-ness and this kind of usefulness, though.

Yeah the multimodal work is already making progress and will likely be the future. That problem seems solvable soonish. Sequence of actions is much harder but with their new Chain of Thought research and whats happened in RL, I wouldn't be surprised if this was mostly solved in < 10 years. Retrieval transformers are the most promising on memory.

I think multimodal work is still very much in the early stages. That said, I don't see any reason why the exact same approach we are using here wouldn't work ultimately for multi-modal settings as well.

Memory also seems easy to solve.

I am still very skeptical of our current approach in relation to sequences of actions, reasoning, discrete choices, and interaction with complex environments. RL is at baby level compared to the advances we have made in language modeling.

The chain-of-thought prompting demonstrates that the output tape can act as procedural memory.

Well it is unfortunately an output that scales at O(n^2) its length, which is not super great once you get to sequences of 1000s of words

Sparse transformers are a thing. Dictionary lookup is a thing. The transformer can probably be trained to store long chains of information in a dedicated memory system, retrieving it using keywords.

These are the things that came to mind in ten seconds. This is not going to be the problem that meaningfully delays AGI.

As I said in another comment, I think memory will be the easiest of these challenges to solve. That said, it hasn't really been solved yet.

If we can scale up S4-style architectures to this size, maybe it will be solved.

Reply on news.ycombinator.com