Anything you can do with a calculator is obviously trivial.
Is there a clear point of departure when AI can no longer handle mathematical reasoning?
Is there a clear point of departure when AI can no longer handle mathematical reasoning?
e.g. IMO 2023:
> Problem 1. Determine all composite integers n > 1 that satisfy the following property: if d1, d2, . . . , dk are all the positive divisors of n with 1 = d1 < d2 < · · · < dk = n, then di divides di+1 + di+2 for every 1 ⩽ i ⩽ k − 2
Although ChatGPT 4 makes a pretty decent start on this already:
> To determine all composite integers \( n > 1 \) that satisfy the given property, we need to closely examine the condition set forth: for a composite number \( n \) with its divisors \( d_1, d_2, \ldots, d_k \) where \( 1 = d_1 < d_2 < \cdots < d_k = n \), the property \( d_i \) divides \( d_{i+1} + d_{i+2} \) must hold for every \( 1 \leq i \leq k - 2 \).
Let's break down the problem:
1. *Composite Number Requirement*: We are looking for composite numbers, which are integers greater than 1 that are not prime. These numbers have more than two distinct positive divisors.
2. *Divisor Property*: The property \( d_i | (d_{i+1} + d_{i+2}) \) means that \( d_{i+1} + d_{i+2} \) is a multiple of \( d_i \).
3. *Systematic Examination*: We will analyze this property systematically for composite numbers starting from the smallest composite number (4) and going upwards.
Let's start the analysis:
- For \( n = 4 \), the divisors are \( 1, 2, 4 \). Here, \( 1 | (2 + 4) \), so 4 satisfies the condition. - For \( n = 6 \), the divisors are \( 1, 2, 3, 6 \). Here, \( 1 | (2 + 3) \) and \( 2 | (3 + 6) \), so 6 satisfies the condition. - For \( n = 8 \), the divisors are \( 1, 2, 4, 8 \). Here, \( 1 | (2 + 4) \), but \( 2 \nmid (4 + 8) \), so 8 does not satisfy the condition.
Let's implement a systematic method to identify such composite numbers up to a reasonable limit (let's say up to 100 for a start) and check if they satisfy the given condition.
The composite integers greater than 1 and up to 100 that satisfy the given property are: 4, 8, 9, 16, 25, 27, 32, 49, 64, and 81.
Each of these numbers has the characteristic that for all their positive divisors \( d_1, d_2, \ldots, d_k \) (where \( 1 = d_1 < d_2 < \cdots < d_k = n \)), the condition \( d_i \) divides \( d_{i+1} + d_{i+2} \) is true for every \( 1 \leq i \leq k - 2 \).
﷽!
Rewrite this post using unicode characters:
> The IMO are a bit more involved...
Answer:
> The IMO are a bit more involved... e.g. IMO 2023:
> Problem 1. Determine all composite integers n > 1 that satisfy the following property: if d₁, d₂, ..., dₖ are all the positive divisors of n with 1 = d₁ < d₂ < ... < dₖ = n, then dᵢ divides dᵢ₊₁ + dᵢ₊₂ for every 1 ≤ i ≤ k − 2
Although ChatGPT 4 makes a pretty decent start on this already:
> To determine all composite integers 𝑛 > 1 that satisfy the given property, we need to closely examine the condition set forth: for a composite number 𝑛 with its divisors d₁, d₂, ..., dₖ where 1 = d₁ < d₂ < ... < dₖ = 𝑛, the property dᵢ divides dᵢ₊₁ + dᵢ₊₂ must hold for every 1 ≤ i ≤ k − 2.
Let's break down the problem:
1. Composite Number Requirement: We are looking for composite numbers, which are integers greater than 1 that are not prime. These numbers have more than two distinct positive divisors.
2. Divisor Property: The property dᵢ | (dᵢ₊₁ + dᵢ₊₂) means that dᵢ₊₁ + dᵢ₊₂ is a multiple of dᵢ.
3. Systematic Examination: We will analyze this property systematically for composite numbers starting from the smallest composite number (4) and going upwards.
Let's start the analysis:
- For 𝑛 = 4, the divisors are 1, 2, 4. Here, 1 | (2 + 4), so 4 satisfies the condition.
- For 𝑛 = 6, the divisors are 1, 2, 3, 6. Here, 1 | (2 + 3) and 2 | (3 + 6), so 6 satisfies the condition.
- For 𝑛 = 8, the divisors are 1, 2, 4, 8. Here, 1 | (2 + 4), but 2 ∤ (4 + 8), so 8 does not satisfy the condition.
Let's implement a systematic method to identify such composite numbers up to a reasonable limit (let's say up to 100 for a start) and check if they satisfy the given condition.
The composite integers greater than 1 and up to 100 that satisfy the given property are: 4, 8, 9, 16, 25, 27, 32, 49, 64, and 81.
Each of these numbers has the characteristic that for all their positive divisors d₁, d₂, ..., dₖ (where 1 = d₁ < d₂ < ... < dₖ = 𝑛), the condition dᵢ divides dᵢ₊₁ + dᵢ₊₂ is true for every 1 ≤ i ≤ k − 2.
I tried an easier problem, extending Rolle's theorem to the multidimensional case with Poe.com:
Prompt: Let U be an open subset of R^n with compact closure K. Suppose f: K → R is continuous on K, differentiable on U, and satisfies f(x) = 0, for all x in K \ U. Show that there exists a in U with grad f (a) = 0.
Poe: To prove that there exists a point a in U such that grad f(a) = 0, we will utilize the concept of connectedness and the Mean Value Theorem.
Since U is an open subset of R^n with compact closure K, we know that K is a closed and bounded subset of R^n. Furthermore, since K is a compact set, it is also a connected set.
Now, let's define a new function g: K → R^n such that g(x) = (f(x), f(x), ..., f(x)) for all x in K. In other words, each component of g is equal to f(x) for all x in K.
Since f is continuous on K and differentiable on U (which is an open subset containing K), it follows that g is continuous on K and differentiable on U.
Consider the function h: K → R defined as h(x) = ||g(x)||^2 = f(x)^2 + f(x)^2 + ... + f(x)^2 = nf(x)^2.
Note that h is continuous on K since g is continuous on K and ||.||^2 is a continuous function.
Now, let's consider the set S = {x in K : f(x) = 0}. Since f(x) = 0 for all x in K \ U, we have K \ U ⊆ S. Therefore, the set S is closed.
...
Stochastic parrot indeed.
Higher level math becomes pretty verbose so enters the domain of language more so than pure symbolic computation. However there is still complex reasoning under the hood. Gpt-4 in ChatGPT flounders when I ask it questions from my thesis, so I’m not sure how it will do with problems it hasn’t seen, where it needs to apply “new” reasoning.
I’d love to know if in your example, GPT is reciting something it’s seem verbatim, or if it is taking multiple sources and combing them.
That's hilarious
Amusingly, the list of the integers <= 100 satisfying the property is correct, and it contradicts itself from the previous paragraph. Maybe if GPT wasn't a one-directional autoregressive model but allowed itself to go back and edit the past, it would have caught up that discrepancy and fixed it - but no such architecture currently exists that would run in decent amount of time.
Given that GPT4 is not a model but a full product, behind the scenes it probably coded up and ran a small python code that translated the problem into code, executed it and got its solution for the first few integers. Which would be a good thing to do to start solving a problem like this, except you're not allowed to do that at the IMO, obviously.
Looking at the output of this program, it suggests that powers of primes could be a class of solution (or maybe even the only solutions? I guess that's all the problem was _really_ asking to prove, but having never qualified for the IMO myself, I can't be sure). In fact, for n = p^k, the divisors are [1, p, p^2, ..., p^{k-1}, p^k], and clearly always p^i divides p^{i+1} + p^{i+2} = p^i (p + p^2). I guess this small remark would have gained me a point at the IMO, only 41 to go ;-)
But the other side of the coin is that having those numbers written down in front of you and not even making a conjecture about powers of prime being the answer would really denote poor mathematical reasoning by GPT4. It's only really proving that it can understand what it's being asked, which, I admit, places it in a better position than maybe 90% of the human population, but unfortunately for GPT4 mathematics is the least democratic science of them all - it's always only the top-1 result that matters in the end.
P.S. Being a former mathematician currently working on deep learning, having (or building!) a model that can solve mathematical questions has always been my dream. I'm not even talking about something that can _prove_ things, even just that can understand and rephrase them in different settings (which in mathematics is very, very hard, even for a human). Or spot weaknesses in already stated down proofs. As a graduate student, having something I could chat about to ask silly question while studying a paper would have been a real game changer. Even for best-of-world professionals it would be useful: when the wrong proof about the ABC conjecture came out, it took months of work from the best minds of our world to read through it and disprove it. If Mochizuki had had some tool for automatically checking his proof (and a smaller ego, I guess) he could have caught that early on, saved everybody a lot of work and the whole world some useless drama.
And while we're closer than ever to reaching that, I think GPT4 is still quite a far way from it. But with the pace we've seen recently in AI evolution, who knows...
Nevertheless, I think the more important question is whether GPT-4 is capable of
1) Listing all solutions less than 100
2) Figuring out the commonalities of the solutions
For 1) I have no doubt that the answer is yes based on its coding skills, in fact it is much stronger at coding than this. For 2) my subjective feeling in playing around with it is it's not consistent at similar problems but it can do it sometimes. Maybe in this case it has seen the list of prime powers <100 so it's very easy for it.
Unless I'm getting myself completely wrong, this also seems to be a very unusually simple problem for IMO's standards. I don't think I ever got myself solving one of them when I tried in the past, but if I'm not fooling myself the most natural approach for this one seems to bring to the solution:
1) Look at the two smallest divisors 1 < d_2 < d_3 of n. Then d_2 is necessarily a prime p, and d_3 is either p^2 or a different prime q. Let's first prove the latter can't be the case: if it where, looking at the biggest 3 divisors [n/q, n/p, n], we'd get that there's an integer a such that: n/q * a = n/p + n. Simplifying a bit, we get (1+p)q = ap, which is impossible because p divides neither p+1 nor q.
2) So, for n not to be a power of p, it must be 1 < d_2=p < d_3=p^2 < ... d_{k+1}=p^k < q. In particular, p^{k-1} must divide p^k+q. However this is also impossible, because p^{k-1} obviously divides p^k, but not q, so it can't divide the sum.
Gosh I feel like a grumpy old man saying "those youngsters, on my days we used to have harder problems than this" ;-)
1)notice it's prime powers
2)notice that 1,p,p^2 eventually leads to contradiction
3)use the last part of the factorization to rule out 1,p,q
I couldn't get GPT4 to do either 2 or 3 even with hints. It's surprising that either due to context length or something else I feel like its reasoning abilities are worse when you try to guide it. But maybe this is true for humans too.
"The condition fails for i=1 since 1 does not divide p+q unless p+q is a multiple of n, which is not generally true" (1 divides everything)
In the second it gives up:
"However, this conclusion is based on heuristic reasoning and examples"
Third attempt it makes this mistake:
"If the immediate next divisor, di+1 , is not a multiple of p (for instance, it could be q or a product involving q), then p does not divide di+1. Hence, p will not divide the sum di+1+di+2 in such a case, violating the condition." (the fact that p does not divide d_i+1 does not imply that it does not divide d_i+1, d_i+2)
Then it gives up again: "However, this conclusion is based on heuristic reasoning and examples"
Then it makes this basic logic mistake: "However, since p and q are distinct primes, p does not divide q, and it's not guaranteed that p divides q+d_i+2 , especially if d_i+2 is not a multiple of p. Therefore, for such n, the condition fails." (the fact that "it's not guaranteed" doesn't imply that it's false)
Overall it proves that prime powers have the property, conjures that non-prime-powers don't, proves that p*q doesn't have it, but completely fails at coming up with a proof strategy that proves that non-prime-powers don't have the property.
I'm more interested in the point where AIs are presenting proofs far beyond human capability. I'm imagining a day when an AI says it has solved some interesting problem and when we ask for the proof it spits out a 4 million page document. What are we supposed to do with that? What's the role of humans in that world?