Ray Kurzweil is wrong: The Singularity is not near
pando.com
pando.com
This is why I believe the singularity will happen in my lifetime. I honestly, genuinely, have no idea whether it really will, but believing that it does makes for excellent ideas, great conversations and a whole lot of excitement (and Judgment Days) in general. I want to believe it. This might be difficult for a certain type of rationalist (that many HN'ers are) to appreciate, but if you really want to believe something very much, the step to actually believing it is very small. Even if it's a conscious process.
In general though, no need to squeeze those butt cheeks so much, this is not the stock market. Fun predictions are fun!
Personally I don’t believe that Singularity will be achieved in the time frame Kurzwell predicts. Especially regarding AI progress I think that we need at least half a dozen paradigm shifts in order to get to the milestone of a truly self sustained AI entity. But you know in some sense believing it gives you an extra motive on pursuing some of these goals and I think that’s the real benefit of this idea. A few visionaries out there like Kurzwell could push science to new frontiers.
Religions do.
Talking a lot about a possible future isn't the same as religious belief.
More seriously, Kurzweil snake-oil selling is just distracting us from serious research in AI. That is the problem. Belief in this woo woo might actually stop us from getting real AI.
My point is that all these "deep" X folks shy away from really hard AI problems which other paradigms are solving more easily.
(Call me when Google measures up to IBM's Watson.)
At least this "shying away" will not unleash the 'AI winter' that "hard AI problem" "solvers" have caused.
Umm. Yeah...
In short, the future we dreamed about 40 years ago hasn't arrived, so I wouldn't bet on the one you think is 30 years away.
I really wonder what would happen. The super-human AI would be able to replace basically all of the jobs. What will people do?
2 + 2 = 5
I want to believe it. Now all my programs are spitting out wrong answers and must be fixed to be in line with this belief. Are we getting the drift of the danger belief poses here yet?
Didn't you read your guide to Ingsoc?
While it's good to have a quasi-official spokesperson who can convey some of the concepts to a broader audience, things also got simplified in a way that seriously distorts them. It's not about arbitrary singular points during our development, it's about the breathtaking directions these developments are destined to go in. That's a much harder concept to sell than, say, a magical tipping point where history ends, but I'd say it's well worthwhile.
The standard singularity talk doesn't reflect what's really happening to us - nevertheless, it's important to open people up to the prospect of continued technological development. It sounds so simple, but this is really not an obvious path to people who believe the future will mainly contain ever-lighter iPads and will otherwise be the same: there will be a couple of points in history where technology reaches the potential to fundamentally change who we are, what it means to be alive, and what our goals are as a civilization.
Technology will make it possible to alter some very fundamental realities humans had to live with for a long time. The impact of this is enormous beyond words. We get to change things in a very big way, including some options that would be hugely unpopular even for the tech-savvy people on HN - like getting rid of material scarcity, for example, or choosing radical life extension.
It's clear that this will be upsetting to a lot of people, and suddenly we need a marketing person like Kurzweil. Personally, I'd prefer Eliezer Yudkowsky would take that role, but that's probably not a good choice for "lay people".
As for whether it's moral, well it depends if the copy is actually sentient or just a really advanced chatbot. And if it is sentient, well we create sentient beings all the time; children.
But I do agree it isn't a solution to death.
Kurzweil is just one advocate. In fact, the idea [1] originated with Von Neumann in the 50s and was popularised by Vernor Vinge in the 90s, and its worth reading Vinge's paper "The Coming Technological Singularity: How to Survive in the Post-Human Era" [2]. Bruce Sterling gave a interesting (and sceptical) talk [3] on the subject a while back.
For what its worth, I'm personally not convinced that we'll ever see a technological singularity because I don't think our culture generates technology that has the required attributes. Siri or Google Now don't need to be self-aware in order to do what they do, and there is no apparent commercial pressure to make them so. And since people build technology for human purposes, the presence of people who sleep, get bored, etc. will act as a drag on any tendency for runaway acceleration.
[1] http://en.wikipedia.org/wiki/Technological_singularity#Histo...
[2] http://www-rohan.sdsu.edu/faculty/vinge/misc/singularity.htm...
[3] http://longnow.org/seminars/02004/jun/11/the-singularity-you...
The development of some kind of strong AI is often proposed as a catalyst for a technological singularity, and I was kind of following that lead. While I agree that self-awareness may not be needed, it is the only thing I can think of that would motivate an AI to enhance itself up the exponential curve. Relying oh humans to do this probably wouldn't work (past some initial pump-priming) as they tend to get distracted by other human-scale things.
On the other hand, speculating about motivation in relation to an AI may be misleading anyway...
I see the future hyperintelligent AI as something more like Iain M Banks "Culture" Minds, but hey, maybe I just can't comprehend it with my puny brain :)
And then extend it again by hooking it up to a co-processor? Still you?
And then keep extending it, until the meat bits are only 5% of the structure, and not necessary for the running of the whole? Still you?
Is You-ness (or identity) fluid and variable, or is it binary - something is either "you" or "not you"?
Yeah.
> co-processor?
What's that mean, even?
A directly accessible computer hooked to your brain, to augment your processing capabilities rather than just memory.
I think it's a wrong question. "You" is in the map, not in the territory. It's how algorithms in our brain factorize the world model.
I think Penrose was on right track with his ideas about brain using quantum effects (our words and comprehension follows rules that are eerily familiar of some quantum effects).
With that in mind, I don't think it's possible to do an easy copy of it without disrupting the way it works.
Probably "still me" in the same sense as the person I was five minutes ago is the same person as the one I'll be in 5 minutes.
"The ship of Theseus, also known as Theseus's paradox, is a thought experiment that raises the question of whether an object which has had all its components replaced remains fundamentally the same object. The paradox is most notably recorded by Plutarch in Life of Theseus from the late 1st century. Plutarch asked whether a ship which was restored by replacing each and every one of its wooden parts, remained the same ship."
There's an early AI researcher who brought up the neuron-by-neuron replacement too, but I can't remember who off-hand (might have been Moravec)
Of course, the question then of who to trust to be your cloud provider running your conciousness-as-a-service process becomes a very serious one. I don't think there's a single company around today that I would trust to run me.
The main problem is the lawyers. https://www.youtube.com/watch?v=IFe9wiDfb0E
If there's one service I don't want to be freemium....
reminds me of "how to get ahead in advertising", or William Burroughs' "Did I ever tell you about the man who taught his asshole to talk" heh..
It's a fascinating question.
(And before you can answer, the clones all respond "yes").
Most neuroscientists believe that there's much more to the human brain than that - there are things going on in our brains like interactions between close synapses using quantum effects. Perhaps we could model that if we understood it better, but the reality is we don't, so we can't really predict whether or not it's something we can copy.
The notion that evolution is exponential is, to me at least, very well supported. This article criticizes the fact that Kurzweil crosses the barrier between biological evolution and technological evolution without mentioning it, and that the utility function used to plot the exponential across evolution barriers is ill-defined. Both observations are true, but I don't think they shake the observation of exponential behaviour in these evolution functions.
The conclusion that this will all end in our minds being uploaded to the ether is more difficult to swallow, I concede. It is a philosophical can of worms, and it is a question that will be left unanswered, even after we can replicate human consciousness in a computer.
However, if you assume there is nothing magical about our brains, that our consciousness is merely the result of (biological) computation, then @AndrewDucker's approach [1] is an easy way to wrap your head around the concept. Instead of thinking about a clicking moment when you get transferred into silicon, think of a continual transferral process, where you get silicon additions, while your organic brain continuously dies. Imagine this process takes ten years. When do "you" stop being "you"? I'd wager: never.
BTW, the same problems you have with brain uploading, I have with star-trek teleporters. If the same machine that can teleport me can copy me, then it is killing me and creating an identical twin. I'd rather not die in transportation, thank you.
What exactly do you mean by consciousness? (I personally don't believe that a turing machine can be consciouss.)
Even if it was (I don't think so), it's possible that consciousness is a passive observer of the universe.
Personally, I define my consciousness barrier at the level of Turing machine running any program that evolves. I also believe consciousness is not binary, it is a continuum. Is a dog a consciousness? Of course. Is a plant a form of consciousness? Yes, at a lower level. A bacteria or a virus? Yes. Sediment rocks? No.
My hypothesis isn't testable. It is also probably wrong. I know that. I am, however, accompanied by any other hypothesis out there :)
I can confirm, experienced this couple of times :)
The way I define consciousness is very simple, it's my current feelings, thoughts and perceptions. It's also the only "thing" that I can be 100% sure about.
I think, therefore I am. However, I also observe what seem to be many consciousnesses. If I am the only provable consciousness, then I also am all the other consciousnesses. How can that be? By the Fermi paradox, I must be the first consciousness. If that is so, the most probable sequence of events is that I (my consciousness) appeared into the the void by pure randomness. Then I got bored, and created all this to pass the time. I am you, and I am every conscious being out there (so, that's why there is this "do unto others" golden rule!)
This is kind of the unified theory of all religions, from mid-eastern origin all the way to Buddhism.
Religion: solved. Now, on to the unified theory of physics.
No, you observe a lot of other people that behave similarly to you. If you define consciosuenss as "behaves like me to a certain extent", than yes, there are mutliple consciousnesses. But consciousness is defined as something subjective, so by that definition, there's only one consciousness. Perhaps you're in a dream and imagining everything. Or in a Matrix and other people are just computer programs.
Consiousness is a different problem. I personally came to the conclusion that a deterministic machine cannot be consciouss.
Is the difference between being conscious or not defined by reading from /dev/random? Most theories of consciousness that point to quantum theory read like that. Unless you assume quantum randomness is god's invisible hand, and then religion infiltrates the argument. (not that God isn't in /dev/random anyhow)
If you accept this view on consciousness, you can conclude that a deterministic system cannot be consciouss.
Of course, you'll probably argue that my view on consciousness is ridiculously unfounded. I disagree, I think there are very strong arguments for it :) A very brief version: (1) There are good reasons to believe that consciousness is more than a complex interaction of physical particles. But physical particles can affect consicousness, so the mind is at least a passive observer. (2) It's actually an active observer, otherwise I wouldn't be writing this. (3) If it's an active observer, the mind must have a way to affect brain's matter.
The idea of the singularity implies that the complexity of technology moves above anything humanity can understand. It is unclear whether there still be any use for our kind of individual conscious mind. At what scale will cognition happen? Will the world itself turn into a kind of brain?
But I'm with you in hoping that the Minds will be like the culture's and keep us around as pets in a sort of enlightened hedonism :-)
There's nothing inherently contradictory of a higher intelligence being composed of many active smaller conscious experiences - arguably this is what the internet already is, just a very slow version of it. We need a better interconnect (and some more durable subprocessors).
The 'cells' in the case of the hypothetical technological singularity will be computer hardware, not organic cells or human brains. There is the tacit assumption that silicon-based AI tech is somehow superior to our own carbon-based 'tech' and will displace it.
Human brains forming into a kind of super-organism would be a different phenomenon. It wouldn't even need AI, or much more advanced technology. It could indeed be argued that this is the case already with the internet and even printing press before it.
I doubt it's superior or will replace carbon-based tech (no quotes here; biology is but nanotechnology that was not made by us), but there are two particular improvements introducing human-designed technology could bring:
- electrical signals in the brain run at 200Hz; minds could in principle work much faster, were they made of something different
- human-designed technology is easier to modify, reprogram and adapt (mostly becasue we built it, so we know how to do it); the idea is that a mind built with it could be made to be able to rewrite/rebuild itself, with luck launching into recursive improvement loop, where a mind redesigns itself to be better, which then goes on to improve itself more, etc.
If you were going to be sedated in a blue-painted room, and then an exact copy of you would be made in a red-painted room, what color of wall would you expect to see when you woke up?
It won't be you but you'll be the only one to tell the difference.
Thinking of the social pressure (from family, work etc.) of "uploading" yourself is more fun.
But the singularity is more than just mind uploading. Technology could advance and people just choose not to upload. You could probably live at least a few thousand years with advanced nanotechnology repairing your brain and body.
What is relevant, and this is the only relevant thing about the entire discussion is:
1) When will we reach the ability to take a snapshot of an entire organism with a nervous system up to a resolution sufficient to satisfy all expected simulation parameters?
2) When will sufficient computing and storage capacity be available to run a simulation off the data captured in step #1?
3) Will the simulation performed in step #2 adequately match the organism as it exists in real life to declare that the simulation a success.
4) When will we be able to perform steps #1, #2 and #3 for a human sized organism?
By the nature of such an experiment, a negative result does not indicate the absence of a positive in the future. And certainly, and inference on a negative outcome at any step cannot be valid without performing the steps, that's just guesswork.
It's also worth noting that inherently estimates on timelines of this are nearly impossible because one cannot exclude the possibility of radical advances in scanning, computing and storage hardware.
Or, if you prefer to have a TL;DR explanation for geeks. Unless you can write and run a program, it's nearly impossible to debug it or make statements about how useful it'll be.
Perhaps never. The major thing Kurzweil is ignoring is that this may actually be impossible.
My information is from http://computer-go.info/h-c/
If you really think about it how far are we from there being ' no distinction... between human and machine or between physical and virtual'.
I specifically think of the 'physical and virtual' stuff. I'm currently having a virtual 'conversation' regarding this topic. None of you who may respond are with me, we're doing this all virtually, and I doubt the time is far off where somebody can have a machine respond coherently with their general thoughts on a subject.
When the author talks about Kurzweil not differentiating between biological evolution and mechanical evolution, I think that our biological evolution has in many ways been a result of our technological evolution, would one exist without the other? Isn't out technological evolution in the areas of food production resulting in our genetic evolution as larger (both taller and fatter) people?
I know I'm picking some pretty minor things here and that means the singularity is still a long way off, but compare the last 40 years to the next 40 years.
Not everything Kurzweil describes to be part of the Singularity has to happen exactly as he describes it. It's a theory, and (from getting only about 1/3 through the book) an interesting one.
I guess haters gonna hate.
If machines ever get the "inventive spark", its seems clear to me there will be rapid expansion of machine influence on the universe. Computation is just silicon (rocks) after all.
So that's the big IF. Can we invent inventive machines? Machine learning is certainly going at rapid pace, NELL etc.