Yes, I believe that, it's part of what I was implying (I believe the LLM weights have some internal representation of math in the same way brains do that allow them to produce proofs)
Yes, I believe that, it's part of what I was implying (I believe the LLM weights have some internal representation of math in the same way brains do that allow them to produce proofs)
I think we differ on what "mathematical intuition" is then. I've seen people that do well in undergrad math degrees simply by massively memorising things and learning how to join them up to some level of degrees-of-separation, but seemingly completely fail to understand, for ezample, why even calculus is how it is. Because they are able to regurgitate the results and "produce proofs" this is never questioned.
The Euclid example also shows my bias towards spatial intuition of mathematical concepts (which is deeply unfashionable) but also exposes exactly where at least current LLMs break down; they do the symbol based pattern matching version, but they cannot leap outside of that, at least today.
If you want to catch them, surely you can find proofs they aren't able to produce.
I used to be a game dev, and one of the interview questions someone came up with consisted of working out the surface area of a variant of Menger sponge to some given level of depth. The bifurcation for people that could do this vs those that couldn't was incredible, and did not follow obvious trends for academic achievement. (The same interview also included the gem "How wide is a pointer?" which also catches a frightening number of people).