Score is based on the interrogator, a human. If you read a Markov chain bot's text you'd guess it was a bot probably 80-100% of the time. With a real human, you'd guess it was a bot maybe 0-30% of the time, depending.

I'm making up these figures, but the point is lower is better, or "more Human-Like". Test was specified as >50% meaning "accurately determined human vs. bot more than half the time". The site claims LLMs are now guessed correctly less than half, which is how the turing test was defined as per the site.

It makes sense, even if you disagree it's significant.

I wonder if there's a hyper-Turing test where an AI passes if the model, itself, cannot determine if it is talking to itself; or perhaps stated differently, maximizing some measure of control and processing duration to successfully conceal its identity under forced processing, discounting a solution that specifically learns to be silent or incoherent. I'm not sure what the value would be, just a passing thought.

This is probably already happening within the parade of censorship systems trying to imbue the models with agency

the Turing test is scored by how often an interrogator can determine if they are talking to a machine or a human. It’s perhaps a confusing way to show it, and leaves out a lot of important information about the result they are citing, but they are saying before the interrogator did better than chance and after gpt the interrogator guesses right slightly worse than chance.

Presumably it means that the human detected the AI correctly less than 50% of the time, averaged over a repeated number of experiments.

From TFA:

>GPT-4 was judged to be a human 54% of the time, outperforming ELIZA (22%) but lagging behind actual humans (67%).

It wasn't even the real Turing test but a lesser version of it. I'm hopeful about uses of this tech, but companies need to be more honest unless they want a second winter.

The incentives don't align with honesty though.

I read that as the pass mark being able to identify human vs machine more than 50% of the time. At 50% it's no better than randomly guessing.