Again, a (bad) pet theory.
[0] Yes, IQ is not a good measure of blah blah blah. I'm just using this a handle to explain things, I don't mean it literally.
More seriously, it seems to be essentially the idea that “surpassing human intelligence” is not the binary outcome many thought it would be, and that much of what passes for human intelligence interpersonally could be imitation of intelligence.
Like, you had thousands of men paying real money to chat with (terrible) bots. To me, that was the passing of the Turing Test. But I know of nearly no person that could possibly fall for that scam. Even family members deep in dementia knew it was a joke. Yet Ashely Madison made a ton of cash.
That, to me, was puzzling. How could it happen that people that are that foolish would be able to hold a job or pay taxes? It made no sense.
So, the (bad) pet theory that I eventually came up with is that human intelligence is a lot wider than we think it is.
The thing that is going to be interesting is now that we have essentially cheap, ethically clear, and realistic digital 'people', what are the experiments that we can do with them and what can we uncover? I'm a little flat-footed even as to the questions that we can ask them now. At the very least, we can use them to 'dry-run' surveys and experiments and have better data collection and stress-testing. Like, you can now generate realistic data now and use that to run the stats while the real surveys are coming in.
Essentially, we have 'kind' and 'unkind' learning environments.
To be successful in a Kind environment, you drill-and-kill. The feedback is near instant and the ranking is clear. These are things like golf, classical music, and chess.
To be successful in an Unkind environment, you learn as much as you can. The feedback is infrequent and the ranking is murky. These are things like tennis, jazz, and business.
I'd think that the compounding interest only plays in the Unkind environments, as you can make new connections on the new data you've got going in. In the Kind environment, new data doesn't make a difference as you're just trying to be perfect at the thing you're focusing on; if anything it's an impediment.
Just that in 5-minute sessions (which is what Turing suggested, not the fault of this study) with non-experts, the conversations seemed to tend heavily towards brief unchallenging small talk - which GPT-4.5 did well at due to many interrogators being poorly calibrated about LLMs being able to speak informally.
I think it might instead make sense to consider the accuracy of the best interrogator/strategy. Most accurate strategy listed in the paper still gets 75% accuracy for instance, and I'd suspect there are many people well-informed of LLM weaknesses that could reliably exceed even that.
Careful. You're smuggling in an assumption that isn't true. Machine don't have intellectual capabilities, and this follows from what the computer as formal construct is. They can simulate the appearance of intellectual ability, as LLMs can, at least in certain respects, but appearance ought not be conflated with cause.
But, if you want, you can replace "some intellectual capability" with "some capability typically associated with intelligence". Ability to solve unseen logic puzzles, for instance.
If you deal in modern machine learning/AI/whatever, you can formulate all sorts of criteria and parameters for an "actually intelligent machine", but it's never going to be as clearcut as "if it quacks like a duck".