Hacker News Reader

Comment by jimbokun | Hacker News Reader

Do we want a "brain in a box?"

Note that this idea is distinct from computers that solve complex problems on our behalf. Creating something with its own desires, ambitions, and emotions, just like us, except vastly more powerful, is there any reason to think something good will result from introducing such a thing into our environment? Does it ever go well for a species when a more intelligent species enters its niche and competes for resources?

As smart as Kurzweil is, I am always shocked that he takes for granted that such a development will be good for humanity. He scarcely gives a thought to the philosophical ramifications of a human being crossing over into a completely digital reality.

I think the question of whether super human intelligence is good for humanity is at least as important as the engineering questions about how to build one.

View on HN

Why is human supremacy a necessity? Why must a child be flesh and blood? If we want our children to be smarter, more powerful, and happier than us, why should we create children as feeble as us?

This is seriously sick, IMO.

I suspect you feel that way reflexively, which is understandable...but not that interesting. However, if you happen to have a more fleshed out (no pun intended) reason for feeling that way, perhaps you could share it?

This is reminiscent of this judeo-christian hate for flesh... That and the fact that our machines and programs certainly aren't our children in any meaningful way.

Overall, I'm finding these singularity fantasies philosophically entertaining, but I'll keep saying that's sick, or disgusting.

Cows are not supreme and look at what the "superior" species does to them. Better to be on top than lower down, as a general rule.

Our machines are not our children.

I know that's a metaphor, but I feel the better metaphor is another species. These machines are likely to be different from us in significant and important ways, and they are likely to have interests we do not share, and the ability to achieve those interests with or without our consent.

if we create them, then they are our children. oedipal perhaps, but children nonetheless.

| something with its own desires, ambitions, and emotions,

There was an SF author singularity panel discussion recently:

http://www.antipope.org/charlie/blog-static/2010/02/what-i-d...

Alastair Reynolds made the great point that the singularity's /identity/ needn't be its own. You could fix it to something / someone else, and have it use its mental power on behalf of someone else. It might not even know it exists.

Actually it sounded great at the time but, surely, if it had a balky human it would make a mental model of the difference between what it wanted to do and what it could do, and reach /some/ conclusion. If it knew about people I guess it could make a good guess. Hrm, I guess it'd be like our own ids, like the apostle Paul said - "I don't do that which I want to do, and do that which I don't want to do". So the creature would come up with a notion of 'singularity frailty' and 'original ineffectuality', perhaps.

For more on this topic, see Hanson's writings on uploads: http://hanson.gmu.edu/uploads.html

These questions are very important, and there are people working on them. Check out the Singularity Institute. The biggest fear would be that we create an intelligent system that doesn't care about humans - for instance giving humans the same moral value we give to grass.

If it can be done then it will be done eventually. So the question becomes who does it first.

Reply on news.ycombinator.com