Hacker News Reader

Comment by brador | Hacker News Reader

Is there a cap on AI intelligence?

Is it just a case of throwing more and more resources at a pattern recognition machine with no upper limit except the energy of the universe and the economy of the algorithms?

Someone else suggested the maximum intelligence is capped as the average intelligence of its training data, which is an interesting take.

View on HN

François Chollet made an interesting geometric analogy on Twitter. We tend to view Intelligence as something unbounded, like the height of a pole. But it could be something closer to the roundness of a polished sphere. Once close to the limit, incremental improvements are less and less significant.

The intuition we have about the nature of cognition and its whole shape and limits is almost entirely built by looking at ourselves and our peers, comparing that to an ant. I feel this is largely naive and misguided.

I've been interpreting AI task solving capabilities as a search problem: being smarter lets us prune more of the tree and try out different things more efficiently. I think this fits into the sphere analogy quite well: at the limit, we need to experiment, use the experiments to create new abstractions and build on them repeatedly. This doesn't necessarily require more raw intelligence, but the time or resources to find how the pieces fit together.

The cap will be reached when LLMs run out of original material to train on and the large majority of their training material is AI generated content.

Reply on news.ycombinator.com