Three arguments against the singularity
antipope.org
antipope.org
EDIT: as a demonstration, I will deal with the first point in the essay. It says: "super-intelligent AI is unlikely because, if you pursue Vernor's program, you get there incrementally by way of human-equivalent AI, and human-equivalent AI is unlikely". Actually, getting to strong AI through math (rather than through mimicking humans) sounds more probable to me. We already have formalisms that can compute the most accurate possible prediction and the most efficient possible way to optimize a a utility function (inferring the right physical laws in the process) if given tons of computing power, for example look up Solomonoff induction or Marcus Hutter's AIXI. These count as superintelligences, or at least superweapons that can destroy the world. Stross's argument does not demonstrate the unlikelihood of someone implementing a fast approximation to AIXI tomorrow.
"First: super-intelligent AI is unlikely because, if you pursue Vernor's program, you get there incrementally by way of human-equivalent AI, and human-equivalent AI is unlikely."
The arguments that follow don't say anything about why it's impossible or even prohibitively difficult to do, they only provide reasons why people wouldn't want to try. There are, however, motivations beyond those he takes into consideration.
Uploading: Again, he doesn't make any arguments as to why uploading is not achievable, he just talks about the very hard ethical questions that arise when dealing with uploaded intelligences. That didn't stop us from inventing nuclear weapons and a host of other ethicaly challenging technologies, so why would it stop us from inventing uploading?
One particular statement he makes in this section I do want to address specifically: "Uploading implicitly refutes the doctrine of the existence of an immortal soul, and therefore presents a raw rebuttal to those religious doctrines that believe in a life after death."
This is so obviously untrue that I don't know how any intelligent person could use it in a serious argument. Religions that believe in immortal souls will simply maintain that the uploaded copy is just that: a copy, a soulless simulation of a real person. How can you possibly prove this one way or another? "Soul" is a purely religious concept, beyond any temporal means of observation or measurement, and therefore not subject to empirical study. I still think you would see plenty of religious people oppose uploading, but for different reasons.
He finishes by discussing the possibility that our entire universe is already a simulation being run in the "real" world, and admits that nobody can prove this one way or another, at least not anytime soon.
This might be true of some religious institutions but not of all. The concept of the 'Soul' is actually derived from Aristotelianism (there is no mention of it as such in the Bible) and many sects, (particularly Catholics) believe that in order for something to be intelligent it needs to have a soul, that is, an immaterial aspect. If an uploaded person, or an AI, or space alien or whatever is demonstrated to be intelligent, it would by definition have a 'soul.'
You would still get lots of folks who refused to accept the person-hood of uploaded persons and those folks might actually dominate, but the actual theological consequences would be different (and far more complex) than you imagine.
Wrong, and wrong. The notion of the "soul" in Greek philosophy predates Aristotle; it's found in Plato's Phaedo, for example-- and there are several words in the Hebrew bible that are usually translated as "soul" (nefesh and ruach, which correspond roughly to psyche and pneuma, respectively.)
I stand corrected on this.
"The notion of the "soul" in Greek philosophy predates Aristotle."
True, but my point is that certain sects of Christianity borrow heavily from Greek philosophy when trying to intellectualize what a 'soul' actually is. Some tend to follow the Aristotelian conception rather than the Platonic (they are distinct.) I'm not trying to say that Aristotle invented the concept of the soul if that's what you're thinking.
That being said, I'll accept your fallacious assumption for a moment and turn your question around: what if an emotional system is necessary for self-aware, volitional intelligence? Our emotions are inseperably intertwined with the other aspects of our minds, and it might very well be that they are a fundamental aspect of consciousness.
Many philosophers and theologians have attempted to make completely rational arguments in favor of the existence of God. My liberal arts theme in undergrad was philosophy so I've read a fair amount, but I have yet to see a compelling argument yet. Could you please make some suggestions?
EDIT: Proving or disproving the existence of God is, in general, a fool's game. One could probably prove that certain types of deities are impossible, but it's impossible to prove that no form of deity is possible. On the other hand, proving the existence of God would almost certainly require a well-documented divine intervention, and even then many people would (rightfully) remain skeptical.
Also, my previous post didn't say anything about proving the existence of God: I said that they made "arguments in favor of religion," which is not the same thing. First, not all religions involve deities. Even if you restrict the discussion to religions involving deities, it is possible to argue that it is desireable or advantageous to believe in God without first proving that God exists. Furthermore, even some atheists argue that religion in general or some religions in particular have had a net positive impact on humanity.
p.s. "...but it's impossible to prove that no form of deity is possible" - could you point to a proof of that?
As for your latter point: if you disprove the possibility of deities with a particular quality or set of qualities, I can then describe a deity which lacks those qualities and is therefore outside the set of deities you have proven impossible. For example, if you can somehow prove that there has never been divine intervention of any form (which I highly doubt is possible to prove), then I could respond by positing a deity that does not intervene in the temporal universe at all, but instead rules only in the afterlife. Since the afterlife is beyond temporal measurement, it is by definition beyond empirical observation, and there is therefore no way you can disprove the existence of an afterlife. By extension, there is no way you can disprove the existence of a deity which rules the afterlife.
b) You are right. AFAIK, religious experiences involves the parietal and temporal lobes, so its not just the emotional system. Agree that it's an important (irreplaceable?) part of our self-awareness. However the fact that non-religious people can act ethically in a harmonious society indicates that it might be reduntant. Still, religious beliefs arise in a brain, so there will probably be a way to turn them off (probably not without sideeffects).
Also interesting is that most of his arguments seem to be economic ones. In short, "it doesn't seem useful so the required investments to develop it will never be made". Not so much that it is impossible physically.
He might be right, he might be wrong, only time will tell. Personally I don't think it is very useful to make predictions about this (which is kind of the same stance that he takes "it's unwise to live on the assumption that they're coming down the pipeline within my lifetime...").
While that'd be a nice thing to have, it doesn't seem anything like an AI in the usual sense... i.e. the sense in which AIs have personhood and their own thoughts and feelings and goals and so forth.
It seems that conceptualizing AIs as human-like is the #1 top mistake that leads people to think AIs are impossible (or, alternatively, safe once they're built). It's more enlightening to think of an AI as a cold mathematical machine that uses computational power to achieve whatever goal the original programmer specified, possibly increasing its own power in the process. It will happily eat your babies if it needs more atoms. Its lack of personhood won't save you.
If a superhuman AI exists, exactly what evolutionary niche are humans going to fill? Because there's a word for species which don't have an evolutionary niche: extinct.
But really, even humans are just cold mathematical machines when you get below a certain layer.
Very surprising coming from a SciFi person - most people who write SciFi (Scientific themed) tend to perform exhaustive research into existing science. Disappointing...
- CS says the path to super-intelligent AI lies through human-equivalent AI (actually, he says Vinge's program says that, and he explicitly criticizes it)
- but I can find an uncomputable perfectly ideal math function
- and I can hypothesize that maybe it'll be possible to approximate it cheaply
- and I can hypothesize that easy instances of it are good enough
- and I will now view human progress as a way of approximating that function, and submit that only easy instances were used.
This isn't merely red herring, it's red herring full of contradictions. If easy instances are enough, what's the relevance of the provably ideal uncomputable problem, besides rhetoric? If humans used easy instances, and they're the only example we know of that achieved I, how is that not weak evidence that AI through easy instances will pass through human-like I?
It shows that there's a path to AI that's gated on solving formal math problems instead of imitating wishy-washy emotional humans. This point may be obvious to you, but apparently it wasn't obvious to Stross.
> If humans used easy instances, and they're the only example we know of that achieved I, how is that not weak evidence that AI through easy instances will pass through human-like I?
Humans are also the only animals that can play chess, but chessplaying software didn't pass through a human-like stage where they got nervous, forgetful, etc. My intuition says math can't be that radically different from chess (both involve search on trees with a complicated evaluation function), and we're only a handful of new insights from having computers decisively beat humans at math. And at that moment, by implication from my previous point, they might just also beat us at everything else :-) Remember that, on an absolute scale, humans are about as bad at math as you can be and still build a civilization (otherwise civilization would've happened at a lower level of development) and also that we probably suck at math because math is a conscious task (see Moravec's paradox).
Related link that should make you scared if you buy my reasoning: http://www.vetta.org/2009/09/monte-carlo-aixi/ .
Enhancements to primate evolutionary fitness are not much use to a machine, - Said who?
I strongly suspect that the hardest part of mind uploading won't be the mind part, but the body and its interactions with its surroundings. That's actually the easy part, the motor and sensory systems even in humans are largely deciphered.
Uploading implicitly refutes the doctrine of the existence of an immortal soul I thought this was refuted centuries ago, and anyway, it's a red herring.
our hard-wired biophilia will keep dragging us back to the real world, or to simulations indistinguishable from it Is the "real world" something stable? The same human brains performed great with both stone age cave living as well as modern megacity-life.
And so on and so on. The basic argument is "It is definately possible, but we may have reservations about it, ergo it's impossible"
b) Stephen Hawking
c) False, soul is not a corporal thing [for those who believe in it], whatever the corpus.
d) We are limited by what we cannot perceive through our brains. We are "dragged back" by our inability to perceive higher intelligence, not because we 're lazy to adapt.
Can you really imagine a group of scientists sitting around a large computer cluster and saying "Well, we could create the first sentient AI and ensure our names are enshrined in the history books, but why bother?"
Games also work as argument against his uploading argument that we are ill-equipped for living in virtual worlds. From all experience so far we _love_ living in virtual environments. There's already enough people which nearly ignore their bodies for hours and live in games like WoW instead.
Climbing mountains doesn't seem that comparable to going to the moon.
Scientists need funding in order to act if it requires many man years.
Thus military uses of AI are likely to be well funded to the tune of hundreds of millions if not billions because they can be viewed as a strategic necessity if it seems like the Chinese could possibly develop an AI before us. But if the general feeling is that a real AI isn't feasible in both Chinese and US circles in the near or medium term future, or there are more valuable/useful near and medium term objectives, there will not be a well funded race to create one.
So yeah, I agree that X might very well be possible (or in widespread usage) in 50 or 100 years, where X is basically anything that seems magical today.
Also, what about Art? What a useless waste of brainpower, yet we keep doing it for thousands of years.
Not transcendence in the Physics/Metaphysics sense but rather in the (at least to our knowledge) fact that dumb matter transcended into aware matter at one point.
I.e. life happened through transcendence from "dumb" matter into aware matter. Pattern recognizing feedback loops with memory that can reflect on our own existence.
I do not have the knowledge to determine the feasibility of the various methods but I do believe that the mind isn't a thing as such but rather a system that simulate a reality, the reality that we experience from our vantage point.
So to me the question is not as much whether we can transcend from one physical form to another but rather how we will connect with more and more external computation that again will mold our internal computation maybe even take over some parts. At least the IO can be replaced (eyes, ears etc.)
But if that can be replaced then what's to stop other areas? Nothing dictates that as far as we know right now.
Somehow the argument reminds me of the Searles (misguided) Chinese Room argument.
The fact is that we don't know whether it's possible. So why not just let those who think it is, work on a solution.
The arguing is already happening. If you enter something in your navigation system and it gives you the wrong suggestions (completions).
Also read up on the filter bubble.
And I certainly don't want to be sued for maintenance by an abandoned software development project.
...let's all hope it never comes to that!
That is, if humans are able to start the chain reaction and come up with something that's able to rewrite its own brain so that it's smarter than it started...a lump of plutonium that's not at critical mass is just going to sit there uselessly spewing out a slow trickle of neutrons, which is more or less what's happening with AI today.
http://www.acceleratingfuture.com/michael/blog/2011/06/respo...
Michael Anissimov Singularity Institute
Yes, and my point is that I don't really care about "not dying". I care about living a long time. Getting hit by a bus (or killed by a roadside bomb) doesn't bother me. Growing old and living for three decades in constant pain and discomfort does.
The time when "Accelerating Change" no longer applies is the same time that the "Intelligence Explosion" and the "Event Horizon" occur or at least that's how it seems to me.
Also, the point of the "Accelerating Change" scenario is that there isn't an intelligence exposion. The "Intelligence Explosion" scenario might or might not have a period of accelerating change before the intelligence explosion, but that doesn't make it the "Accelerating Change" scenario.
Tell that to the German green party, who despite clearly wanting a carbon-neutral environmentally friendly economy are in the process of closing down Germany's nuclear reactor fleet.
(Etc.)
But if, as some people suspect, it's something that can be solved by setting the right set of clever algorithms to work, running on twenty-years-out hardware, then the ethical problems that the public has with the endeavor will be irrelevant. Someone will just do it, ethics and regulations be damned, especially given the monumental potential rewards.
There are arguments that suggest that the actual algorithmic complexity of the "software" running on our brains is comparable to moderately complex software that humans routinely develop (even though the computational and "RAM" requirements of the "brain algorithm" are enormous), so I wouldn't discount the possibility that a small research group might be able to do the job without any serious level of financial support...
Physiology has as much to do with intelligence as electricity generation and distribution has to do with computer function. We humans are actors in a society, and that's how we measure intelligence. All that's needed is to be able to respond to social cues. Turing had it correct from the start. Don't get it twisted.
Misleading. People are against abortion because how else are you going to punish those terrible terrible women for not staying at home and doing what their fathers and brothers tell them?!
Now, feel free to debate with them whether that is true or not, but don't lie about their reasoning.
That said, please, please, please keep these kind of topics off of hacker news. I've never seen abortion end in anything but a flame war.