I rather like this paper, but I think it is generous to say their benchmarks measure "intelligence".
[0] https://arxiv.org/pdf/1911.01547
We have no dang clue what intelligence is, nor how to measure it.
[0] https://arxiv.org/pdf/1911.01547
We have no dang clue what intelligence is, nor how to measure it.
- Sam Altman[0]
"Can you define intelligence?"
"Yes, it is this many moneys."
[0] https://www.businessinsider.com/sam-altman-ai-utility-electr...
Broadly in laymen's terms, no, we don't. We do have some ideas though. We have lots and lots of different tests of different facets of intelligence. How high can you count? (crows can count up to 30), how long can you remember things? (elephants can remember things from up to 50 years ago), Memory games, Reasoning. "The Science of Human Intelligence" , by Haier, Colom, and Hunt. is a good read.
my very naive question to him back then was how close we were to understanding human cognition. we were both fans of grand strategy games (though the few hundred hours of Stellaris I played vs his thousands in EU4 paled in comparison) and I was asking if it was possible to map human cognition to the same array of interdependent logical chains-of-reasoning that games like that could be boiled down to
his answer, in short, was 'we are so, so, so far away and no, that's, at best, a reductive mental model of intelligence'
I keep that conversation in mind whenever I hear about all this talk of AGI - that realistically we're so far away from actual AGI in the same way that the inventor(s) of the wheel were from a gas-powered car, and there's many paradigm shifts to go in how we even understand what the nature of intelligence is before we get there
https://pmc.ncbi.nlm.nih.gov/articles/PMC5230747/
roughly, it reviews common techniques in neuroscience, and comes to the conclusion that they would not be able to understand even simple computing platforms that we have perfect information for (and can perfectly stimulate any internal connection, can perfectly read out the values on any internal connection, etc).