Hacker News Reader

Comment by skywhopper | Hacker News Reader

Where did the 30% requirement come from? Sounds like something the contest organizers added to make it possible to "pass" without fooling 2/3 of the judges. Using a young teenager as a character also seems like a cheat unless they had other 13-year olds to be the judges. The character needs to be a peer to the judges. Most 13 year olds behave oddly enough in the opinion of most adults that it's got to be easier to credit weird behavior to generational or cultural differences. So kudos to the winners on strategy, and boo to the contest organizers for having such poor rules.

View on HN

Most likely from here:

>It will simplify matters for the reader if I explain first my own beliefs in the matter. Consider first the more accurate form of the question. I believe that in about fifty years' time it will be possible, to programme computers, with a storage capacity of about 109, to make them play the imitation game so well that an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning.

COMPUTING MACHINERY AND INTELLIGENCE

— A. M. Turing

http://loebner.net/Prizef/TuringArticle.html

Is he talking about 109 bits? Because that seems pretty impossible to me.

> As we have mentioned, digital computers fall within the class of discrete-state machines. But the number of states of which such a machine is capable is usually enormously large. For instance, the number for the machine now working at Manchester is about 2 165,000, i.e., about 10 50,000. Compare this with our example of the clicking wheel described above, which had three states. It is not difficult to see why the number of states should be so immense. The computer includes a store corresponding to the paper used by a human computer. It must be possible to write into the store any one of the combinations of symbols which might have been written on the paper. For simplicity suppose that only digits from 0 to 9 are used as symbols. Variations in handwriting are ignored. Suppose the computer is allowed 100 sheets of paper each containing 50 lines each with room for 30 digits. Then the number of states is 10 100x50x30 i.e., 10 150,000 . This is about the number of states of three Manchester machines put together. The logarithm to the base two of the number of states is usually called the "storage capacity" of the machine. Thus the Manchester machine has a storage capacity of about 165,000 and the wheel machine of our example about 1.6. If two machines are put together their capacities must be added to obtain the capacity of the resultant machine. This leads to the possibility of statements such as "The Manchester machine contains 64 magnetic tracks each with a capacity of 2560, eight electronic tubes with a capacity of 1280. Miscellaneous storage amounts to about 300 making a total of 174,380."

Indeed it should be 10^9. Likely an OCR error - you can find several other 10^9 properly formatted somewhere else in the paper.

And don't forget that the word bit was only around two years old at the time the paper was written. (I have no idea when Tukey first suggested it. I'm using Shannon's publication for a date.)

10^9 bits is only (check my math) about 120 MBytes.

So Turing was perhaps a bit optimistic there?

That's almost certainly meant to be 10 to the 9th.

More likely 1E9

30% of the time, five minute conversations, a "simulated" 13-year old? At least "the conversations were unrestricted".

The Turing test is not a test. Instead, it is an operational definition of "intelligence", a very slight formalization of the idea that something is intelligent if it seems to be intelligent.

As a test, it obviously has to have some kind of limits like this "competition", but as soon as you put limits on it then it stops being useful and becomes both gameable and meaningless. The Turing test has already been passed, long ago, if you have limits suitable to the Doctor or Parry.

> Instead, it is an operational definition of "intelligence"

Exactly. It's a straightforward formulation of what a strong AI would be capable of. It makes no sense to have a restricted Turing test that can be passed by a useless chatbot. It means absolutely nothing.

Not at all. If you've been following this, the age of the test has been getting older and older for the past 15 or so years. They're not resting on their laurels at 13-year-old, next year maybe they'll make 14, and the year after that...

It's a way of having a better metric than pass/fail. This is the real problem with AI. Everyone expects that AI research should go: you put a bunch of programmers in a room, and they work for a few years and build an AI.

The best intelligence production we have is human child-rearing. This process has always taken 15-20 years, and is backed by millennia of research. Assume you have a digital computer capable of human-like thought. Without an example of a computer capable of learning faster, it stands to reason that raising the computer into an AI capable of conversation should take 15-20 years.

Of course, one thinks there would be a way to do it faster, but on the first go?

I assume you mean weak AI. And I don't see how you can dismiss the meaning of passing this test so easily. Although it's a trivial example, I can definitely see such chatbots being applied for spamming purposes, which by definition exploits hapless victims.

No, I think that's a correct use of strong AI.[1] "Strong AI" is work towards general-purpose intelligence, at least as sentient, conscious, intelligent, or whatever term you prefer to use as you are. "Weak AI" is work towards usefully solving problems that would previously have been thought to require general intelligence, such as chess programs or self-driving cars.

[1] Although usage may be changing. Pity.

Weak AI is when a computer can (appear to) act intelligently, strong AI is when a computer can actually think (Russell & Norvig). Therefore, passing a Turing test would be a textbook example of weak AI, because you only need to simulate intelligence well enough to fool the judge.

Like I said, usage is changing.

I studied AI shortly after the "AI Winter", which is where I got my definition. Strong AI---working towards a general intelligence---was strongly out of favor, especially with funding agencies. It still remains so (but see Watson). But solving limited problems heuristically (or statistically) that are not otherwise algorithmically tractable (a loose translation of "would appear to require general intelligence") has always been a fertile field.

Turing's argument, which is a philosophical argument, is not meaningful if you put any limits on it---time, topic, behavior (really, Parry is better than the Doctor), which is why it is better thought of as a thought experiment. If you have limits such that it can be gamed, then yes, it is fair to say "you only need to simulate intelligence well enough to fool the judge." Which makes it uninteresting.

But the question is, if you can "simulate intelligence" well enough under any conceivable circumstance (and yes, all actual human beans will fail here), how can you say that it cannot "actually think"?

The Turing test is not a definition of intelligence. Turing explicitly suggests the test in order to replace the question of the definition of intelligence. The test is most definitely a test, in the sense that it provides an estimate, rather than a clear cut definition.

While the limits in this case were admittedly rather strong, this does not form a fundamental objection to the test. The bar can be set progressively higher.

What is the limit, the ultimate height of the bar, then? A test that requires three score and ten years with a 50% success rate? Longer? The problem with the Turing test as a test is that it is always possible to game it, to create a system that can pass with any set of feasible limits without being able to do anything else.

The problem with positively defining intelligence is that all such definitions seem to end up begging the question and are therefore unsatisfactory to somebody. Which is what makes the Turing test philosophically interesting.

I don't know about limits, you can decide for yourself what it would take for you to be convinced some agent is "intelligent." The fact that we've been trying for 70 years is completely immaterial. How long until we got the ability to fly? Seems like we had been trying at least a few thousand years, that says little about feasibility.

The test is an adversarial game, and both sides can attempt new strategies of fooling or seeing through the opponent. I think your claim about being able to pass any set of limits is rather bold, and I would be curious about arguments for it.

We're already in the territory where there are certain applications for these chatbots, and there is no fundamental argument why we this line of research could not progress further until it reaches its goals. I don't mean to say I believe we'll have AI soon, just that you can't fault the Turing test for not doing what it should.

What do you mean when you say there is a problem with positively defining intelligence? Do you think we should define it negatively? There is a lot of criticism of the Turing test, but alternative proposals are exceedingly rare.

You can also pass the test this way by having "generous" judges contributing to the 1/3, which is likely because the judges are not impartial: they are emotionally invested in being part of a positive result. I wonder how Kevin Warwick himself voted, for example.

A more correct test (which admittedly doesn't cover this issue) would be to give each judge a conversation with one human and one computer, and for them to say which one they believe is the human.

>to give each judge a conversation with one human and one computer, and for them to say which one they believe is the human.

I always assumed this was exactly what the Turing test was about. Guess I was wrong.

This is indeed what the Turing test as originally proposed is about.

A blogspam linking to this page also claimed the alleged "boy" was described as a Russian boy to whom English is a second language. The official report from University of Reading doesn't mention this, can anyone shed more light on this?

http://gizmodo.com/this-is-the-first-computer-in-history-to-...

Reply on news.ycombinator.com