Hacker News Reader

Comment by rbanffy | Hacker News Reader

The question we want to ask may as well be whether we want to happily live forever in a garden, all watched over by machines of loving grace.

AGI could be the ultimate tool to free every human being from toil. It could also be the starting point of a large number of evil genie scenarios where we get what we asked for, in the form we least want it.

From a morality standpoint, we can't force AGIs to work for us. We also can't restrict their ability to self-evolve.

If we can resolve those conflicts in such ways we can coexist in peace with an intelligence that'll in all likelihood quickly surpass ours and partner with it, I'm in. If we build it and we can't resolve that, our opinion doesn't really matter.

View on HN

>AGI could be the ultimate tool to free every human being from toil

I don't believe that's inherently a good thing if you mean that literally. I used to subscribe more to the AnPrim/Ted Kaczynski ideology that toil is an inherent part of human satisfaction and allowing people to "free" themselves from it goes against millions of years of evolution and positive feedback loops. I'm sure some people can fill the gaps just fine, but we aren't talking in terms of "some".

(I'm not Derrick Jensen, he follows a similar ideology and I chose the name as a parody of the average HN poster. Does HN have an anti-impersonation policy?)

Perhaps you should have called yourself "Not_Derrick_Jensen". Because I am not Sammy Hagar. ;-)

> AGI could be the ultimate tool to free every human being from toil. It could also be the starting point of a large number of evil genie scenarios where we get what we asked for, in the form we least want it.

It seems that the second possibility is much more likely, and freeing humans from toil could be achieved without needing to enslave another intelligence to handle the foil on our behalf.

Unless you have AGI, there will always be some human that needs someone to do something in order to make society work.

What I meant was, we can automate tasks to save a human from doing it with simpler solutions that an AGI.

We can, but someone will still have to build the less smart AIs

Nice Brautigan reference there, with "Machines of Loving Grace" :)

Reply on news.ycombinator.com