Hacker News Reader

Comment by tenaciousDaniel | Hacker News Reader

I'm not well versed in this area, but from my perspective, I see this as the fundamental problem:

Every action my brain takes is made of two components: (1) the desired outcome of the thought, and (2) the computation required to achieve that outcome. No matter how well computers can solve for 2, I have no idea how they'd manage solving for 1. This is because in order to think at all, I have to have a desire to think that thought, and that is a function of me being an organism that wants to live, eat, sleep, etc.

So for me, I just wonder how we're going to replicate volition itself. That's a vastly different, and vastly more complicated, problem.

View on HN

It isn't hard to give an AI a goal but it is hard to do so safely. As a toy example we could design an AI who treated say, reducing carbon emissions as it's goal just as you treat eating and sleeping as yours. The issue is that the sub goals to accomplish that top-level goal might contain things we didn't account for, say destroying carbon-emitting technology and/or the people that use it.

Human's have many basic goals that are very dangerous when isolated in that way. It seems to me that nature didn't care (and of course, can't care) if it was dangerous at all when coming up with intelligence. Maybe we shouldn't either if we want to succeed with replicating it.

Worrying about some apocalypse seems counterproductive to me.

I agree that there's some aspect of volition, desire, the creative process...whatever you want to call that aspect of human thought that seems to arise de novo.

But speaking of de novo, I'm not at all sure that a desire to think a thought it required in order to think. The opposite seems closer: the less one tries to think, the more one ends up thinking.

I'm pivoting from your point here, but I see that bit as the hurdle we're not close to overcoming. We are likely missing huge pieces of the puzzle when it comes to understanding human "intelligence" (and intelligence itself is not the full picture). With such a limited understanding, a replication or full superseding in the near future seems unlikely. Perhaps the blind spot of the experts, as /u/leftyted alluded to, is that their modest success so far has generated a reality-distorting hubris.

It's like the more advanced stage of "a little bit of information is a dangerous thing".

If I'm remembering my terms right, "embodied AI" is one theory or group of theories about interaction with an environment creating the volition necessary before generalized AI can be created.

There's active research in Model-Based RL right now that tries to tackle 1) and 2) together.

Reply on news.ycombinator.com