By 2029 no computer or “machine intelligence” will have passed the Turing Test
longbets.org
longbets.org
Conversation is actually a rather poor measure of intelligence. I would say, show me a computer that can learn anything that it wasn't specifically programmed to learn. This doesn't mean unsupervised categorization or learning to play a video game. I'm talking about a scenario where programmers present their code or machine with no knowledge of what the task will be. Not something chosen from a known list of possibilities, but any task that a human could conceivably be taught to perform in less than an hour. Anything from writing a sonnet in iambic pentameter to assembling ikea furniture based on instructions. A true test of _general_ intelligence.
I would take that long bet out to 2129, and beyond. I don't see software with that level of intellectual flexibility being written in our lifetimes, or the lifetimes of our children or their children.
[0] "While it is possible to imagine a machine obtaining a perfect score on the SAT or winning Jeopardy..."
[1] "... a skeptic about machine intelligence could fairly ask how and why the Turing Test was transformed from its origins as a provocative thought experiment by Alan Turing to a challenge seriously sought."
Looking just now at Turing's paper he actually deals with Sonnets and suggests the conversation might go:
>Q: Please write me a sonnet on the subject of the Forth Bridge.
>A : Count me out on this one. I never could write poetry.
Now that I could probably do.
The sad thing is I happen to know humans who would utterly fail at this (which, I know, is not invalidating the fact that a computer passing this would be worthwhile, but care has to be taken as to what that means), routinely challenging my faith in mankind.
The Turing test is simple because it is easy to add complexity to something that is simple. As a human, I can convince myself that an idiot is a genius and a genius is an idiot because of introspection. Humans have imagination, and we very often don't see how that warps our perception of intelligence. We still think we can be the distant, disconnected, scientific, observer - carefully constructing a pristine and universally objective technology. But computers, out of all kinds of technology, show us how much of our humanity goes into our creations.
There are also a number of assumptions when approaching problems such as composing a sonnet or assembling furniture. With a human, you assume that they have a broad understanding of the language you are communicating in, that they have some knowledge of poetry even if just in passing, have experienced cadence, rhyme, or other constructs and have prior experience of the world from which to draw 'inspiration'. You also assume that they have seen and used a variety of furniture, tools, and even just generally explored a physical environment to a degree where they are capable of manipulating objects by moving, rotating, stacking, joining, etc. or even as basic as having the concept of object permanence. They must also know all of the relevant nouns and verbs referring to the pieces and tools they will be using.
If you continue to think along these lines you very quickly realize that the only reason you can teach a human being something in an hour is because they possess years or decades of previous instruction, programming, and prior learning attempts as well as a data set of visual, aural, tactile, olfactory, and spatial experiences encompassing a similar time frame.
You said anything we can teach a human within an hour: do you mean a fresh human that's only just been supplied with a brain? Because you can't teach a baby anything besides basic stimulus response on that time frame, and I'm pretty sure we can get computers to do that if we rig up the stimulus response hardware in a way comparable to a newborn.
What I think you actually meant, and what I think is a completely rigged test, is comparing a human with years of training and adaptive hardware modification to a computer with absolutely no training in its ability to learn a new skill or building out its knowledge base. That, of course, is a completely terrible comparison.
I'll take your bet, and call it by 2050, if you're willing to compare a human with 5 years of hardware and knowledge base training and adaptation to a computer with 1 year of hardware adaption (eg, FPGA circuit reprogramming) and 1 year of knowledge base training.
The "within an hour" part refers to the test itself.
Having said that, I do not believe that passing the Turing test necessarily a desirable property of an AI. At the very least, a AI would need to be able to lie about its childhood. Furthermore, a AI may be truly alien, so that even if we assume a truly sentient AI it would necessarily need the ability to hide its own internal states, for example a AI may view the entire sex/gender/procreation topic as mere curiosity, and instead emulate a Human well enough that it can fool other humans.
But why would an AGI individual have to be able to lie about it's childhood ? It could simply have a childhood. But I understand your argument : "Human" AI is constantly trained for the function "be a human". Machine intelligence is being trained with "is human X cheating", "is this spam", "what is the french for apple", "which way will stock X go" (and other "what will <large number of humans> do in situation X" questions) ... functions.
Keep in mind that your own brain is also simply running an algorithm, with "hidden state". Here's the first thing it hides pretty well : there's no single "you", there's about 300 regions in your brain, that are physically different, in different locations, and not directly linked. And if the rest of your brain gets disabled, each of those 300 regions can use your body to convince your own mother it's really you she's talking to. And the function your brain is executing is not to "have a soul" or something like that, but is mostly a behaviour-copy algorithm. It searches out other humans, observes their actions, learns to predict them and then uses what it's learned to "be you". Most people will not believe this is how it works, not even when they know quite a bit about neurons, and know pretty well that this is what our neurons do.
But when it comes right down to it, you, me, everyone is an "AI algorithm" just like Google search. Aside from the shear scale, number of connections mostly, there is nothing all that remarkable about the hardware it's running on.
I love how an AGI algorithm is presented in the follow up to the Battlestar Galactica series, "Caprica". Zoe Graystone explains it at one point in the series. It's fiction of course, but still. Her algorithm searches the internet, your computer, and everything and anything it can find for pictures and videos of it's "target" and then builds the function "given situation X, what would <human> do ?" and then puts the algorithm in a virtual body that looks like the human, and uploads it to V-world. She runs it on herself, a friend and accidentally on a third person she doesn't know and then proceeds to get herself blown up by a friend/terrorist (they can be both at the same time, you know) in the first episode, and their avatars wake up in V-world, sort of a facebook virtual reality edition, unaware of what has happened (and as V-world is mostly used for sex, dancing and debauchery it's not that easy to find out unless you're looking for it). Currently we don't have nearly enough data on any person to make this work ... but I am pretty convinced it is possible to make this work.
This would work, since it's doing the same thing your brain does when it hears the question "do you remember feeling happy when your mother held you when you were a baby ?". First, your brain doesn't actually remember that either way. It simply works it's way through to a good answer to that question. The thing it'll do is predict how you should answer that question, given (mostly) how other humans answer that question.
This is a very different problem from "lying about its childhood", specifically it's much more solvable. Can I predict how humans would answer questions about their childhood ? Certainly. Easy. Make the algorithm take enough contextual information into account (so that it both realizes what realistic answers are and either answers realistically or responds in a way that humans respond when they don't have a "good prediction" for an answer, for instance by saying they don't remember, or by changing the subject, or ...)
If you have young kids you will see this process in action. You have to "teach" kids memory. When they talk only a little, if you ask them something about earlier that day, they will respond. And the response makes sense, but it's not actually what happened. You will at some point realise this and correct them. Correcting them teaches them to match up what happened with their response, because they learn to predict that that's what's expected of people. Over time this correcting them will work, and they will have learned memory (and then they learn to lie, especially on questions like "did you hit your sister ?", and when that ends badly they learn to "not lie", like adults do. "Did you hit your sister ?" "She stole my lego").
Note that the underlying memory is also a prediction. If I ask you what happened this morning, and you want to try to answer correctly, you will start with an obvious fact (which isn't necessarily the truth), like "I woke up". Your brain will take that thought, and predict (not remember) what happened next. So I woke up, ok next, brushing teeth, ok next, read cell phone messages, ok next, ... ah ! the answer to the question.
People assume a conversational partner is 'real'. I could write "Hello? Is anyone there?" on a postcard, and it would 'pass' in a chance encounter sort of way.
The Turing test is useless without the tester demanding cooperation. Otherwise I've got an AI for sale that will recite an eloquent speech about the Brooklyn bridge. I call it PRINTFriend.
As far as I know, research in those directions is nowhere near as sophisticated as Kurzweil tries to make us believe. The mathematical models for neurons I've seen may reproduce some firing statistics, but they are not at all suitable for actually modelling behavior of a system in response to a stimulus.
However, Kurzweil's argument focuses on the fact that he believes we will someday be able to simulate the human brain, and that's something I disagree with strongly.
The problem is in completely understanding it. If we completely understand it, then it shouldn't be hard to simulate.
The important part of the Turing Test isn't whether or not we can build a computer to 'pass it', it's what is it that differentiates us from just being complex computers that can spit-out the right answer when asked, and if there actually is any difference. The question really is "If a computer can act exactly like a human, to the point where people can't tell the two apart, what exactly is the difference between that computer and a real human?". Most people would say that the computer can't "think" and a human can, but if the computer can shift bits around in such a way that it comes to the 'right' response, what's the difference between that and 'thinking'?
When the machines can do that then we can compare apples to apples.
Thinking is a nice feat and I think it's going to be solved in many of our lifetimes. It's definitely something I'd like to research more at some point.
WE are the terminators - so... yeah.
But who equated them?
And then, the word 'fundamentally' has been thrown in to bluff confidence, but without providing actual supporting reasoning. That's a bit like some of the old ELIZA evasions.
Based on the content, I vote the parent comment 'AI'. Did I get it right?
(And no, contrived scenarios with computers pretending to be foreign children don't count[0])
[0] http://blogs.wsj.com/digits/2014/06/10/did-eugene-goostman-p...
Now I don't have any idea if he will be correct, or even what a machine that will pass the Turing test will be like. But if you accept the premise that technological advancement is on an exponential curve, half way in terms of progress will be much closer to 2029 than half way in time.
(Note I am not saying that technology is exponential or that it is following Moore's law or that it will be exponential forever or that 2x every 18 months is fact, its simply an extension of Kurzweil reasoning to its logical conclusion.)
Exponential growth of technology hasn't solved machine intelligence (by direct simulation) for essentially the same reason why it hasn't solved numerical weather prediction or made quantum computers pointless.
To expand on weather prediction: you need an order of magnitude improvement in computer speed to make weather forecasts with the same quality one day earlier. So exponential growth of computing power means linear growth in terms of subjective benefit.
To be fair, we don't understand the difficulty of modeling intelligence anywhere near as well as we understand the difficulty of modeling the weather. But, given the limits of our current abilities to simulate the brain, it seems reasonable to guess that similar principles could hold.
Talking like a human is a bounded problem, it has no margin for growing (or shrinking). Any kind of growth in capacity will help against it.
You can not claim that exponential growth on space travel tech isn't enough to get intergalactic travel. Any non-declining rate of progress will get us there eventually. (Whatever "exponential growth" means on this context, for computation it's very well defined. Also, whatever realist is on non-declining rate of progress, the rate for computation is clearly increasing.)
Sure they are! They reduce it to a problem that's only polynomially hard! :-p
>Exponential growth of technology hasn't solved machine intelligence (by direct simulation) for essentially the same reason why it hasn't solved numerical weather prediction or made quantum computers pointless.
Okay, but that's a rather high bar to judge it against. Humanity has solved many lesser machine intelligence problems, like web search, voice recognition, routing, and recommendation. Though I agree the relevant exponential growth was in the power of the algorithms, not so much the hardware.
It's great progress, but it is also a far cry from what humans can do and there is no clear path at all to get there - currently.
Massive progress is being made in natural language understanding too. A number of papers have just come out on using deep NNs to do question answering or conversation.
http://arxiv.org/abs/1503.02364
Here is some other relevant work of recent:
http://arxiv.org/abs/1502.05698
http://arxiv.org/abs/1410.3916
https://www.reddit.com/r/thisisthewayitwillbe/comments/2zi8i...
https://www.reddit.com/r/thisisthewayitwillbe/comments/25a4r...
https://www.reddit.com/r/thisisthewayitwillbe/comments/307pt...
http://www.noahlab.com.hk/topics/ShortTextConversation
http://allenai.org/content/publications/hixon_naacl_2015.pdf
I mean, for some particular, limited version of "exceeding", yes: http://www.wired.com/2015/01/simple-pictures-state-art-ai-st...
Anyway this paper has a method for addressing this problem: http://arxiv.org/abs/1412.6572
Similarly, I imagine we will get very good domain-specific "AI" in the next ten years: some sort of amped-up Siri that can, say, answer questions about hotels in a travel destination and book one for you.
I do not think we will get a chatbot that can hold a conversation on any conceivable topic at a level that you would confuse for a human.
When it started getting attention a few years ago algorithms sucked at it. No one would have believed in a few years we could beat humans on it. It seems like a case of moving goal posts, where every time a milestone is reached, it's disregarded.
I think it's sort of a case of AI researchers tried moving the goalposts closer, and now we're moving them back. Back in the 60s, a bunch of people thought a general-purpose artificial intelligence was a realistic goal in the near future.
Then it turned out that problem was hard, really hard. AI researchers instead defined-down the problem. Instead of building AI, they worked on building "chess-playing AI" or "traffic-optimizing AI". These are useful and interesting in their own right, but they're not really what people mean when they talk about "intelligence".
Over time, this meant that the entire field of "AI" became discredited, which is why nobody talks about AI anymore, they talk about "machine learning", or "deep learning", etc. If somebody says they're going to make a human-level general-purpose AI, they're no longer taken seriously.
All I'm saying is that AI is rapidly advancing. I'm not claiming its human level yet, and I don't really care what people in the 50s predicted.
Anyway it will happen pretty soon and then people will just say it wasn't a good test.
I mean for one thing the bet is just as much about the turing test as it is whether or not you believe this foundation is going to exist in 2029.
It also seems like they take all the bet money and then invest it while they're waiting to pay out. Seems like a pretty sweet deal.
"The Long Now Foundation itself is the brainchild of inventor and engineer Danny Hillis, who launched the non-profit to build the clock. The Long Now foundation has over 3,300 members who are supporting the project, but Bezos is by far the most prominent and seemingly deep-pocketed, kicking in a projected $42 million"
I hope they'll be with us in 2029.
We actually host some related bets and predictions, including:
"The original URL for this prediction will no longer be available in eleven years." http://longbets.org/601/
"The Long Bets Foundation will no longer exist in 2104." http://longbets.org/137/
Investing the money is definitely part of what makes this interesting. Thanks to compound interest, some truly large sums could be at stake by the time the bet is resolved. (We also couldn't do it any other way, as a number of the bettors are unlikely to be around when the bets are resolved.) The money, though, is not held directly by the Long Now. It's in a special account set up with The Farsight Fund of Capital Research and Management Company. That's mentioned here: https://longbets.org/about/ http://longbets.org/faq/
As to whether it'll exist in 2029, I'd say the odds are good. The Turing bet is a 27-year bet and we're already nearly halfway there. But if you don't think so, I will be entirely glad to bet against you. ;-)
The Long Now for years had a little museum space that got only a modest number of visitors. But conversation about the long term is their goal, and they realized that coffeehouses and bars are where a lot of good conversation happens, so they converted the museum into a cafe during the day and a bar at night.
If you're ever in San Francisco, it's a great nerdy tourist stop. It's in Fort Mason, on the north edge of San Francisco between Fisherman's Wharf and the Golden Gate Bridge.
What are you trying to say, and can you say it in English?
That modesty, though.