1960s chatbot ELIZA beat OpenAI's GPT-3.5 in a recent Turing test study | Hacker News Reader

1960s chatbot ELIZA beat OpenAI's GPT-3.5 in a recent Turing test study

arstechnica.com

I really really doubt that an AI from the 60s is at the same level than gpt 3.5

I don't doubt it. ELIZA isn't programmed to say "I'm sorry Dave, I'm afraid I can't do that" While ChatGPT is loaded to the gills with cautionary programming like that.

Yeah, the title is misleading here. What happened is that the Turing test judges, who had to decide whether they were talking to a human or an AI, judged ELIZA's output as much too poor for current AI, and therefore probably generated by a human intentionally being uncooperative.

This is almost exactly what the article says.

> ...some interrogators reported thinking that ELIZA was “too bad” to be a current AI model, and therefore was more likely to be a human intentionally being uncooperative.

That's really funny

A good example of how misleading a test result can be when the test tests the wrong things.

Wonder if this would hold for GPT 4, and if GPT training on the tuning/human feedback was for more 'conversational' responses. Since recently, it does seem like GPT 4 is a little more 'dry' and guarded in its responses. A few months ago it would give me some sassy answers, now it is more business like.

"Surprisingly, ELIZA, developed in the mid-1960s by computer scientist Joseph Weizenbaum at MIT, scored relatively well during the study, achieving a success rate of 27 percent. GPT-3.5, depending on the prompt, scored a 14 percent success rate, below ELIZA. GPT-4 achieved a success rate of 41 percent, second only to actual humans"

GPT-4 did better. I think the article stated 3.5 was like that by design.

Yep missed this " And like GPT-3.5, GPT-4 has also been conditioned not to present itself as human"

Actually most of the 4 is better. Wonder why 1 is not.

Reply on news.ycombinator.com