Hacker News Reader

Comment by BoiledCabbage | Hacker News Reader

One of the things sci-fi really seemed to get right is that we will have AGI long before we'll have agreement that it is actually AGI.

People will keep finding some small case or reason why not to call it AGI. And then finally once that last case is knocked down and we have agreement on a definition, we'll realize we crossed that threshold a "long" while back.

And I'm not saying we have AGI now, just that it's now clear to me how this process will play out.

(Where "long" in AI development timeliness probably doesn't mean the same thing "long" meant even in the 2010s.)

View on HN

I think the last hurdles will be sentience and a theory of the mind. Theory of the mind is probably something that a LM could predict given enough data, but sentience? I don't think we understand enough about our own sentience for us to create it in a machine.

LLMs can already predict theory of mind in text and keep good track of character motivations and knowledge in a story.

It goes back to the old question of whether matter is fundamental and consciousness emerges from it or the other way around. My inclination is the latter, by way of Descartes. "I think therefore I am." If you can think "I am me" then you know with certainty that your first person identity exists but all other information that comes to you through your senses will always be an incomplete picture of the material universe. I think this is the strongest argument why consciousness precedes matter (or pervades through all matter, depending how you look at it). If sentience is a fundamental property of the universe, the way I see it, all matter shares the same soul. That makes it perfectly logical for a machine that mimics the pathways of the brain to behave just like a brain, to me.

> I don't think we understand enough about our own sentience for us to create it in a machine.

Invention preceding understanding is the norm, not the exception. We created fire before understanding chemistry, and we constantly use pharmaceuticals without really understanding how they work. Invention first, then theory comes along to explain and generalize.

For all we know, sentience is a necessary side-effect of semantic processing of any kind, in which case LLMs already have a form of sentience.

So yes, you're right that we don't understand our own sentience. In fact, we understand so little that it could literally be staring us in the face right now and we don't realize it.

It may be that we found out that sentience is not as magical or unique as we originally thought.

Some people claimed that "creativity" (another rather nebulous term) was a uniquely human trait, and machines could perform tasks that require this. But recent generative AI models have started to make people question that position.

Neither of those are required for intelligence.

Reply on news.ycombinator.com