Imagine we trained an AI on everything ever written, but the catch is we’ve restricted the training data to the year, let’s say, 400 BCE and earlier (Ignore the fact that most of what was written then is lost to us now and just pretend that’s not issue for our thought experiment) The AI is also programmed to seek new knowledge based off that starting knowledge.

Also pretend that this AI has an oracle it could talk to that would help the AI simulate experiments. So the AI could ask questions and get answers but only in a way that builds ever so slightly off what it already knows.

Making any progress at all in this experiment and discovering new knowledge is what we’re after and “new knowledge” would be defined as some r that’s demonstrated using some p and q as propositions, where r is neither p or q, and r is also correct in terms of 2025 knowledge.

If the AI, with the aid of the knowledge it started with and the help of the oracle to let it ask questions about the world and build off that knowledge, can ever arrive, or exceed, 2025 knowledge then it’s at least generally intelligent and equal to a human. Although the bar could maybe be even less.

It loses, however, if it never advances, gets stuck in a loop, or in some other sense can’t make progress.

This is intelligence: to proceed from things everyone agrees on and ask questions and formulate assertions that depend on propositions holding true, and in the process demonstrate new things that were not already part of common belief.

I don’t know how this experiment could be done in real life with an LLM for example but this is a story version of what I mean in my original comment.