Paul Allen: The Singularity Isn't Near (2011)
technologyreview.com
technologyreview.com
First, he points out that Kurzweil is extrapolating, and extrapolations can be wrong. This seems obvious. And it doesn't do much towards Paul's "isn't".
Second, he says that software and hardware will have to keep improving, and that might not happen. Again, this seems intellectually equivalent to saying "you might be wrong". No evidence is provided that progress will slow.
Third, he says that the singularity will require either a bottom-up biologically inspired model, or a non-biologically inspired "AI" system.
The complexity of the former will take a long time to overcome. Since Allen concedes that sufficient computational power is already here, he seems to be arguing that it will take over a hundred years for us to have a detailed model of the human brain. Looking at the progress of science, this seems terribly conservative, yet little justification is offered. Allen's posited "complexity brake" seems positioned tendentiously — why are we going to start hitting it now, instead of fifty years ago?
As for the AI route, he writes: "But when we step back, we can see that overall AI-based capabilities haven’t been exponentially increasing either, at least when measured against the creation of a fully general human intelligence." What does it even mean to be exponential relative to a binary variable? He is right to say that current methods are limited, and don't achieve generalized intelligence, but offers no reason why that won't happen. His argument seems equivalent to simply saying it hasn't happened yet.
This piece offers little intellectual contribution, and the only reason we're reading it piece is because of the author. It reads like he started from the conclusion he wanted, and threw some paragraphs in that direction. His argument, essentially, is that he's betting against continued progress in science, software, and hardware. That just seems crazy.
Given that Paul Allen donated $100 million in 2003 (and pledged an additional $300 million in 2012) to the Allen Brain Atlas[1] and Allen Institute for Brain Science[2], he probably knows more than most people about this. He certainly isn't betting against continued progress.
[1] - http://en.wikipedia.org/wiki/Allen_Brain_Atlas
[2] - http://en.wikipedia.org/wiki/Allen_Institute_for_Brain_Scien...
They would view Allen's response as "not delusional" rather than pessimistic. I'm often surprised by how widespread singularity fans think belief in it is.
Outside the tech industry, where people have spend their entire lives in a field that has seen no increasing exponential returns (other than problems solved purely by computation, which they consider a wonderful thing but often not a fundamental advancement to their field) there is very very little belief that "the singularity is near" or that kurzweils utopian interpretation of the singularity is correct.
There's every reason to believe that things aren't going to get faster or better--CPU development seems to be petering out compared to the gains of yesteryear. Worse, the products that drive those sales, and the markets that drive those products, seem utterly uninterested in anything beyond simple apps and hardware-as-appliance.
The thing a lot of people seem to forget is that the market is what drives this stuff, and that in the absence of market forces not very much happens. This is the world we live in.
This is arguably true if your primary metric is single-threaded performance on branchy-integer-type code. Admittedly this is what normal users see most of the time, which is why consumer desktops and laptops haven't been very exciting the last few years. But the cost to build a petaflop supercomputer keeps dropping.
Doesn't matter how fast you think we will overcome this problem, it illustrates to many people that the intersection between the world that is not increasing exponentially and chip speeds starts to matter a great deal when you want to actually solve a problem by moving floating point numbers about.
As an example, what comes out of Siri & similar AI programs over the next ten years will ultimately be viewed as a leap, driven by mass consumer demand. But Siri only exists due to thousands of small progressions that might have seemed trivial by themselves.
It's very hard for me, as a programmer, to take this idea of a singularity seriously. I know how all of the technologies that Kurzweil is banking on work at a very low level and I call bullshit that this is ever going to happen. It would require a type of software that simply does not exist yet, something akin to a self-programming software, and we are so far away from that that it might as well be cold fusion.
I think it's much more valid to call the "singularity" the point in time when technological expansion started occurring at an exponential rate. Thus I would put the singularity at about 200 years in the past, right around the time the spinning jenny was invented, and right before the industrial revolution. Now there's a point in time that I can point to and say "something meaningful happened". This is complete pie in the sky stuff, and I'm extremely cynical that it's not just something that Kurzweil talks about to sell books and conference tickets.
> I think it's much more valid to call the "singularity" the point in time [...]
Isn't this just redefining the word "singularity", and so making the discussion about something entirely different? So it might be better to qualify that with "industrial singularity" and the one under consideration here is the "technological singularity".
As far as I can tell the only way to become popular as a futurist is to lay out predictions for the future in far more detail and with far more certainty than could every be justified.
(imho, more convincing than Allen's).
However the brain's design information is the genome, the laws of physics and a universe.
You only have to look at protein folding, and the complexity of the resulting molecule and how that interacts with the rest of the world, so see how much complexity is there outside of the genome - and it's that complex end result that you have to simulate or replicate.
He goes onto say that "..the amount of design information in the genome is about 50 million bytes, roughly half of which pertains to the brain. That’s not simple, but it is a level of complexity we can deal with and represents less complexity than many software systems in the modern world."
As above, this is greatly simplifying the end result that you're trying to replicate - the genome is just the software that's running on the OS/hardware of the universe. To replicate that virtually we'd also need a virtual universe.
To simulate a brain, you get to do that for a few hundred trillion proteins that make up a brain...and all their interactions with all the other trillion proteins.
The complexity doesn't come from the proteins so much as the physics that govern the proteins.
And while the translation of the genome into a brain is
not straightforward, the brain cannot have more design
information than the genome
We do need to understand in detail how individual types
of neurons work, and then gather information about how
functional modules are connected. The functional methods
that are derived from this type of analysis can then
guide the development of intelligent systems.
The Google self-driving cars (which have driven over
140,000 miles through California cities and towns) learn
from their own driving experience as well as from Google
cars driven by human drivers.
~
All the buzzwords and nerd feelgood talking points are there, but I'm very put off by the random anecdotes and clumsy assertions.
It can't have more state. The initial conditions have just as much state as any later generation. The initial state is {off, off, off, off, off, off, ... }. That's just as many bits as any later stage.
What you really mean is that the initial state is easily compressible. (Run-length encoding can express an entire empty grid in two numbers as length, value.) And that's pretty much what Kurzweil means, actually the Game of Life is a great analogy here. The genome is a compressed data set, which doesn't express or exhibit anything useful in itself, but can be decompressed using external inputs of organic matter and energy to procedurally generate the neurons that will form the intelligence.
That encoding is only viable if you restrict the game to a finite grid.
In any case, the amount of information in the game of life is merely (initial state, number of iterations) which is certainly finite for a finite board.
> How do we get on the order of 100 trillion connections
> in the brain from only tens of millions of bytes of
> design information? Obviously, the answer is through
> redundancy.
> Combining human-level pattern recognition with the inherent
> speed and accuracy of computers will be very powerful.
The reality is that most progress follows a log normal curve (http://en.wikipedia.org/wiki/File:Lognormal_distribution_CDF...); it starts linear, and ends flat. Some times it starts flat, looks like its going exponential and then goes flat... but it always ends up flat.
The trick is, does the singularity happen after we hit the flat curve (which means it's going to be some nebulous time in the future that may never arrive) or does it happen while we're still on the 'exponential acceleration' curve (which means it may happen in-this-lifetime).
I don't see us on an exponential increase path for cpu power, cost reduction or algorithmic strength at the moment.
If anything I see us as passed the main hump of growth and into the 'slowdown' phase for all of these.
So, I'm dubious of this rebuttal, which is basically endorsing the stupid 'growth for ever! exponential growth! yay! growth!' idea. -_-
Pi can be described by a relatively small formula/function. It can generate very long sequences of seemingly random digits. The complexity in all of those infinity digits is still just that of the simplest formula that generates Pi.
Pi is a great example.
The full length of Pi cannot be compressed using traditional means into the same number of bytes that the algorithm that generates Pi can exist in.
The point is that a function can generate a dataset that is complex, without redundant information in the data. A dataset that cannot be reduced to a smaller dataset.
The idea that a complex function can only generate a simple data set with 'high redundancy' is obviously nonsense.
My point being that the OP is suggesting the DNA that is the functional generator for the brain generates highly redundant physical structures because there is a limited amount of information in the functional generator (DNA).
While the DNA may generate redundant or repeated structures, it does so (if it does) because of the function. Not because of the information density of the DNA.
As your example of Pi shows, a trivial information density can generate complex datasets without repetition.
The existence of redundant data can be proven by the ability to store the same information with less data.
Pi's information density is in fact no larger than the small function that generates Pi's digits -- even if traditional compressors fail to identify it.
A classical example is the laws of thermodynamics [...]
Somehow I cannot bring myself to think about the whole idea (of singularity, economics of emulated minds, etc) seriously. Whenever Robin Hanson (of Overcoming Bias[1]) writes about it, I usually skip the page, even though his ideas are new and insightful. I also read some stuff from Eliezer Yudkowsky (of Less Wrong[2]) and again it's a great concept, he even wrote some cool stories about it[3], but despite all the emphasis on correcting own biases and other follies of the mind, feels way too much like some sort of techno, cyberpunk religion from the 80s.
[1] http://www.overcomingbias.com/tag/future
Yes, hardware technology has followed an exponential rate of improvement, and there's no obvious reason to believe that will stop.
But software certainly hasn't. Programmers today still deal with the same issues they dealt with 30 years ago. Separately, we've figured out some cool pattern-recognition techniques, but there's absolutely nothing indicating "exponential" growth in how smart our programs are.
Yet the "Singularity" depends primarily on software, not hardware. And all this talk about getting around it by just "scanning in" actual brains and simulating them on hardware... well that still doesn't show how those simulated brains are suddenly going to get smarter than our own.
(And then you've got a whole lot of human-rights and personhood issues when you start dealing with actual seemingly conscious brains running on silicon, with real childhood memories, real emotional desires and whatnot -- I mean, they would basically be actual people, not some kind of abstract AWS brainpower cluster...)
http://bits.blogs.nytimes.com/2011/03/07/software-progress-b...
But all he (and the article) is talking about is speed improvements to algorithms. Singularity-type AI is not about the speed, and it's not about complexity in terms of number of moving parts either (a million-line code base might be larger than a 10,000-line one, and more complicated, but not necessarily any more conceptually complex).
What matters for the "Singularity" is conceptual improvements. We're still writing brittle computer code that crashes a compiler upon a single slightly misspelled constant name or missing comma, where the intention would still be crystal-clear to any human programmer. I don't see any kind of "exponential" progress whatsoever relating to the fundamental building blocks of artificial thought, which is what the entire "Singularity" premise is based on.
I am prepared to go out on a limb here, as I have done before, and say that business and research cycles that involve standard-issue humans are incompressible beneath a certain duration - they cannot be made to happen much faster than is possible today.
Kurzweil's Singularity is a Vingean slow burn across a decade, driven by recursively self-improving AI, enhanced human intelligence and the merger of the two. Interestingly, Kurzweil employs much the same arguments against a hard takeoff scenario - in which these processes of self-improvement in AI occur in a matter of hours or days - as I am employing against his proposed timescale: complexity must be managed and there are limits as to how fast this can happen. But artificial intelligence, or improved human intelligence, most likely through machine enhancement, is at the heart of the process. Intelligence can be thought of as the capacity for dealing with complexity; if we improve this capacity, then all the old limits we worked within can be pushed outwards. We don't need to search for keys to complexity if we can manage the complexity directly. Once the process of intelligence enhancement begins in earnest, then we can start to talk about compressing business cycles that existed due to the limits of present day human workers, individually and collectively.
Until we start pushing these limits, we're still stuck with the slow human organizational friction, limits on complexity management, and a limit on exponential growth. Couple this with slow progress towards both organizational efficiency and the development of general artificial intelligence, and this is why I believe that Kurzweil is optimistic by at least a decade or two.
I've always wondered that. I think I could substantially reduce my carbs but I love sandwiches.
Any ideas?
But it is so enjoyable and useful to creatively extrapolate or generate history, if only to encourage us to sink our time and energy into pushing that edge.
The superintelligent machines will take over the world and probably destroy humankind as a side effect. Consequently, they think that we urgently need philosophical musings over possible ways to ensure that the inevitably created superintelligent AI overlords would be build using principles that make them friendly to humankind. This is (according to them) the only hope to save humankind from extinction in near future.
Then, such gems like "The Singularity Summit is the premier conference on the Singularity. As we get closer to the Singularity, each year's conference is better than the last."
They seem kinda obsessed with "thought leaders" and not so much "thought doers."
Edit: I guess Singularity Institute http://singularity.org/ since the banners and text seem to match up, maybe?
In any case, the news of the transfer of the summit is evidence against your last point. It means (I hope) that Singularity Institute is trying to free up their researchers to be able to concentrate on actual work, rather than organising a conference every year.
If we are truly accelerating technologically, why have the exponential gains in wages and PPP in the developed world largely stopped? They have hardly kept up with inflation.
Compare 1891 - 1931, 1931 - 1971, and 1971 - 2011.
People point to improvements in computer technology -- why have they not yielded significant improvements in the world of stuff? There has not been the expected productivity increase at all.
The question is, if you subtract computers, why do you not see any improvement? Is this not evidence of a technological slowdown, at least in the world of stuff?
Getting aside from the pseudo-religious arguments which are often raised in objection (and, personally I believe legitimately), I think we are left with 2 options:
1) the Singularity is a real thing which will happen at some point in the future
2) Consciousness is not able to be crafted by man and the logistics of downloadable consciousnesses and infinately extendable lifespans are forever beyond us, and life will continue pretty much as it has since pre-history, with better technology making a 'richer' life a possibility for a greater and greater proportion of humanity.
As much as I would hope for option 1) the question I feel needs to be asked is why have not other conscious beings (which logically must exist elsewhere in the universe regardless of the numbers you plug into the Drake equation) come to us in their flying robot suits?
The complex interplay of neurology and computing is only in it's infancy, and I wait with bated breath at the advances we are making in our understanding..
Assuming it's true, the main conclusion from Fermi's paradox (which is really an observation more than a Paradox) is that there is some kind of "cliff" that vastly reduces the number of civilisations that achieve the technology required to come visit us with their robot suits.
The interesting question is, where is this cliff: before where we are now, or after?
If it is before, then that's great for us. What it means is, for example, that perhaps the evolution of life, or multi-cellular life, or animal life, or intelligent life, etc - is so unlikely that even though there are trillions of trillions of attempts, those that succeed are so far apart that they will never meet (perhaps thanks to the expansion of the universe, or through the difficulty of travelling across interstellar distances, etc). That's the lucky scenario.
The unlucky scenario is that this cliff is after where we are now. Perhaps there are millions of intelligent species even in just the Milky Way, but perhaps intelligent life is doomed to self-destroy eventually (for example by reaching a Singularity, building a Dyson sphere, and then basically disappearing up its own arse; or perhaps by ending in some kind of nuclear war, or biological wipeout after someone's biology experiment goes horribly wrong, etc).
That latter scenario would mean that we're likely to kill ourselves before we get to the stars.
I hope for the first scenario.
Edit: Of course, as pixl97 points out, perhaps Fermi's paradox is an incorrect observation, and the aliens are already here, they're just being really cautious about being observed.
Bacterial life appeared on Earth pretty much as soon as the planet had cooled down.
An argument could be that some extremely unlikely chemical coincidence was needed to create life, but I think the observation gives more support for the theory that if you have a planet with prebiotic soup, the biotic part is going to kick in pretty soon.
But then it took 4 billion years -- or 30% of the whole universe's lifetime -- for life to invent multicellularity (with some important steps, like the invention of nucleus and thus eukaryotic cells, still unicellular life, half-way in between).
I don't have a problem believing that bacterial life is abundant in the universe. But life on Earth, inventing multicellularity only in 4 billion years, and then sentience only in 0.6 billion more (multicellularity obviously being the harder part), is among the fast ones.
Wait another 4 billion years to give the slower ones a fair chance to discover multicellularity, too.
I've read a paper which pretty much lays out exactly the argument you describe, and the whole time I couldn't help but think the author was ignoring the very obvious. We can't even be sure that advanced civilizations would use radio that much, if at all, and even if they did, it's not as though we've conducted such a thorough search that we can confidently say "nope, no radios here". Not even close, in fact.
1. We are the first, or we are the first to get close to the singularity in our light cone.
or
2. Generating energies or over coming the challenges of even traveling close to the speed of light is hard.
or
3. Space is big. You just won’t believe how vastly, hugely, mind-bogglingly big it is. I mean, you may think it’s a long way down the road to the drug store, but that’s just peanuts to space.
or
4. They have.
Think a bit about this one. Let's define singularity as the ability to either a. upgrade our brains without any biological limits or b. upload our consciousness to a computer that can be upgraded without limits, as immortality and all other consequences will follow from this. Interfering with a pre-singularity race would be the equivalent of us using a newly discovered limitless brain enhancing technology to bring monkeys, dolphins and even cockroaches, as from our infinitely high level downwards a monkey would seen close in intelligence to a cockroach, to human and then limitless superhuman levels! Think about this for a moment! Would you want to waste your technology to "upgrade" inferior life (and possibly trespassing their "free will to choose their own evolutionary strategy"), when you could create new intelligence from scratch that would be designed to evolve faster and be more like you so you could feel empathy for it?
I don't think a post-singularity civilization would bother contacting a pre-singularity one. They would either analyze it stealthly without interfering ...or just use them or their homeworld for resources without caring about their fate or even their survival!
Now, assuming there really is a discernible point of "singularity" by the a. and b. mentioned above, the meeting of two post-singularity civilizations would be a whole different thing. I think we have quite a bit to "grow up" until we get invited into any "galactic club" :)
2. It doesn't matter; the galaxy can be completely converted to computronium in around 10M years even at 1% of the speed of light.
3. Exponential growth is bigger than space.
4. ...or in a simulation.
Some people think that space is expanding as well. These people think that intra-galactic distances are relatively consistent, but inter-galactic distances are increasing in an accelerating fashion.
"Additionally, the expansion rate of the universe has been measured to be accelerating due to the repulsive force of dark energy which appears in the theoretical models as a cosmological constant. This acceleration of the universe, or "cosmic jerk", has only recently become measurable, and billions of years ago, the universe's expansion rate was actually decelerating due to the gravitational attraction of the matter content of the universe. According to the simplest extrapolation of the currently-favored cosmological model (known as "ΛCDM"), however, the dark energy acceleration will dominate on into the future."
Jason Silva has an explanation that I think could be on the right track: http://www.youtube.com/watch?v=nQOyJUDTKdM&noredirect=1
Anyway, the one good thing that could come out of the Singularity discussion is maybe we can have a concerted effort to make it happen. A lot more research money into big science, for example. Hypersonic flight, maglev trains, the Texas supercollider, space exploration, medical research, clean energy, etc. Imagine if the the first flight happen 100 years before the Wright Brothers, or the telephone had been invented even 50 years earlier. We might not have a Singularity by 2045, but we can actively accelerate our quest for knowledge.
Back in the 20th century, every looked at the incredible advances in travel technology from 1850-1950 and assumed the growth curve would just keep on going. But it didn't; it peaked out somewhere in the 1970s. Since then it's gotten cheaper, but not faster.
I think the Singularity is the exact same sort of projection. Progress in computer technology was incredibly fast from 1970 to 2000. My phone has a processor thousands of times faster than my first computer, back in 1982. And if progress continued at that rate, the Singularity probably would be inevitable. But it hasn't. My desktop computer today is not significantly more powerful than the machine I had in 2007. My laptop is much better than my laptop then, but it's only on par with my 2007 desktop. My phone is insanely better than my 2007 phone. The current trend is for computing power to get smaller and cheaper, rather than getting more computing power. That's great, but I don't see it getting us to the Singularity anytime soon.
I think it's a mistake to compare specific devices from different eras, like a 2007 desktop and a 2012 desktop. In order to find total computing power, you need to add your phone, tablet, desktop, laptop, and cloud services together. When you do that, you see that we all have far more computing power than we did in the past, but we've chosen to spread that power over a variety of different devices.
Your desktop may not be getting much faster, but your TV got smarter, baby toys have more computing power than your first PC, a new car has more smartphone features than your phone from 2007, and everything else seems to be getting more computing power whether it needs it or not.
The Big Science stuff is cute and all, but don't for a second pretend that technology in common use isn't mindblowingly awesome.
http://en.wikipedia.org/wiki/Motorola_StarTAC
Everything discovered in CERN today could have been discovered 20 years ago.
http://en.wikipedia.org/wiki/Superconducting_Super_Collider
Yes, because of semiconductor advances (Moore's Law) computers and communications advance at a blinding pace. However, we would all benefit if we could push other areas too.
It's not about how far we've come, it's how far we have yet to go.
If you could quantify social progress, I would theorize that it would be a linear growth. Which means, our capability, and how it fits in our society are becoming decoupled from each other at an increasing amount every year.
Evidence: The increasing uselessness of human beings to economic activity.
Evidence: The callousness to economic externalities with biological consequences of existing suprahuman organisms. i.e. Monsanto, Dupont, Keystone XL, the US Government
Conclusion: The singularity would most likely result in the complete extinction of biological human beings.
Nota Bene: Structured like a High School debate contrapositive because that's what the singularity always sounds like.
Emacs was first released in 1976.
Lisa with its mouse and GUI was first developed in 1978.
The first handheld mobile phone was demonstrated in 1973.
The TCP/IP protocol was first standardized in 1982.
Thirty years on we're still relying on the same core technologies, though more mature.
What new technologies have appeared in recent years that will become more mature in 5, 10, 20, 30 years?
I'm sure I could find a lot by Googling, but asking this group seems to be a more efficient way to benefit from some curation.
At the neuron level, we are still at very immature levels of understanding, but the path and direction being taken looks very promising.
I feel like we might be there today. I'm not sure what could be engineered tomorrow that would surprise me meaningfully.
If someone says that they have a working fusion reactor, I'd just be like, "Geez, finally. What the hell took so long?"
Without clear definition one can say that the Singularity already happened.
Chess masters definitely are considered intelligent. So why computer programs which beat them aren't called super intelligent? Because they use brute force algorithms?
http://lesswrong.com/lw/vb/efficient_crossdomain_optimizatio... tries to address questions like the ones you ask. (Obviously the definition proposed there of "efficient cross-domain optimisation" isn't the definition of the word "intelligence", because no such thing can exist, but it is a definition that seems to match with our intuitions well.)
However, intelligence can also be defined as "the ability to maximize a risk/reward function." In this case, Kurzweil and Allen aren't referring to a specific function, so the first definition supersedes this second, more precise definition.
Economic growth is faster than exponential. We see increasingly rapid growth at any point in time: pre-Cambrian vs. post-Cambrian evolutionary, pre-mammalian vs. mammalian, evolutionary vs. paleolithic, pre-agrarian vs. agrarian, agrarian vs. industrial.
I don't think we're going to see a "Singularity" in 2045. We might see 10-50% annual economic growth by then. That wouldn't surprise me. At this point, we're probably making serious in-roads on a wide variety of health problems, and life expectancy at birth will probably be over 85 and may be undefined. I think it's a good bet that someone born in 2045 will see 3000, not because of a Singularity, but because such a person won't even begin to experience old age until the 22nd century.
Economic growth is NOT faster than exponential in a sustained growth steady-state. Do some googling for long-term economic forecasts, and you will find much research that supports low single-percent growth for the 21st century [3]
[1] http://en.wikipedia.org/wiki/Unified_growth_theory
[2] http://en.wikipedia.org/wiki/Malthusian_trap
[3] http://www.economist.com/blogs/buttonwood/2012/11/economic-o...