AI Is Throwing a Roadside Picnic
metedata.substack.com
metedata.substack.com
That’s the whole issue: it’s not knowledge, it’s data. AI can not provide an account for the proofs, or be questioned about them, or organize a talk exposing the idea because there’s no understanding behind any of it.
This is what Tao has been saying; that the results, in the strictest sense, were never the goal. It is ultimately the understanding (that previously was required to form a proof) that is valuable.
For somebody deeply familiar with a topic and all the nuance around the problem and approaches that didn't work, I'd be quite surprised if they couldn't cajole it into explaining what finally did work in terms they understand. Unless it turned out that these proofs were just well beyond anything humans were anywhere near being able to figure out, in which case, no great loss that the AI figured it out because we were nowhere near it.
To humans? To check if the explanations actually make sense we would have to pause the new AI in while it's rediscovering the result and steer it with the explanation to see if it arrives at the result faster or slower/same.
It very well may be that explnations need to be further explained until they don't sound like gobbledygook to humans. If that's harder than arriving at the result then it's a sign of how far ahead AI actually is.
But understanding to what end? I'm if it's for the pure aesthetic experience of engaging in the beauty of math than that's fine, but that's typically not the view from my experience.
The goal of understanding is to advance the field, since historically human understanding was the only way to do this.
For the vast majority of human history, the world we experienced was a strange and unintelligible space. Humans survived at the by the grace of forces beyond their control. For a brief window of time human existed in a world where they increasingly saw a relationship between their understanding of the world and there ability to steer their environment towards a desired outcome.
I believe now we're seeing that that was a historical anomaly. The world is once again entering a state where it becomes increasingly unintelligible to us. Nick Land already famously pointed out, many decades ago, that the global financial system is already a form of "artificial intelligence" we don't really control. We're seeing that spread wider and wider.
The world making sense peaked around 1960s and has continued to breakdown ever since.
That said, getting a point solution and not knowing the path to get there, is certainly much less useful, and generalization is much harder. Again, more work for mathematicians, but maybe without the easy "I solved it" proof metric for rewards. Goodhart's law strikes again.
I think you have a point, but this is the reality we have to face. The genie is out of the bottle. Even if OpenAI stops publishing AI-generated proofs, it's only a matter of months before tools with similar capabilities are in everyone's hands.
Hopefully, this genie will grant us a few wishes
Mathematics is only useful to the extent that we can do something with it, and I would guess that still requires human mathematicians to understand it.
The human mind has been a black box, and largely still is.
So we traded one black box for another, in more than one way. The new black boxes are going to be able understand and keep up with their own work, much more quickly than we will.
The "alien" picnics, in this case, are a prelude to settlement and expansion, and we have only met the children.
Your point on understanding mostly holds (although our own mind's inscrutability but usefulness is a strong counter-case), but it will increasingly work against us.
The same is true for traditional computer programs. The standard argument against valuing understanding is "well, you don't understand the assembly, processor microcode and the electrical signaling when your program runs either, why do you care about understanding the high-level code then?"
I'd counter this by saying that high-level languages have (mostly!) self-contained abstractions that allow you to "understand" a program deep enough to make accurate predictions of its runtime behavior, answer "why" questions when errors happening and safely make changes to it - without having to understand all the stuff at lower levels.
In contrast, with AI, I see the risk that we lose understanding of both the low- and the high-level structure of things. Then we really end up in "unpredictable alien artifact" territory.
- https://en.wikipedia.org/wiki/The_Evolution_of_Human_Science
- https://gwern.net/doc/fiction/science-fiction/2000-chiang.pd...
Meanwhile, the Rubin observatory is creating an archive that grows by 20 terabytes each night, and nobody really worries about this. Dealing with that amount of data is definitely a problem, but also a nice problem to have, provided that there's a scientific community that can deal with it. Mathematics has unexpectedly become Big Science and there needs to be a better relationship between the funders of the scientific instruments and the scientists themselves.
It's a bit confusing that the output from the scientific instrument looks like badly-written academic papers. Maybe it should be treated more like raw data? The format is not ideal, but historians (for example) often have to deal with far worse.
The 2 books I’ve mentioned are very international on the contrary.
BTW, “The kid from hell” is also really good.
That is why I find the prospect of spoiling the plot without warning to be bad taste and didn’t finish the article.
It has no capacity to think, argue, demonstrate, or certainly do anything even remotely close to “intelligence”
I'm not saying that OpenAI's results are good or bad. But I can say that at least humans could try to understand them and get not only the discoveries themselves but some techniques that could help other progress down the road.
It's fair to ask whether we'll be better off as a result of AI Math. But many people are failing to see any sort of positive outcome and I think this is close minded and borderline irrational.
I don't blame Mathematicians for trying to protect their careers. Anyone would to the same in their place. But the rest of society should take a hard look at what's happening and, as the financiers of mathematical research, make an independent assessment of whether humanity needs "old school" mathematical research at all versus a model with a lot more AI.
The book never tells us that this is actual trash, it's just a theory. If the theory is true however, the aliens might also not understand the damage they've done, it's just a bit of littering to them. Or they don't care, they just destroyed an ant hill, so what.
The author of the article might have a bit more faith in the current generation of AI than I do, I don't believe that LLMs are really capable of creating that big a gap. The discoveries they make are all based on human knowledge and understanding, so we can comprehend the foundations on which they are built. Now if models can train and iterate on their own, to a much greater extend, then maybe, some day, in the far future their discoveries could leap frog humans to the same degree as the aliens in "A Roadside Picnic" and not understand that their output is damaging to us. That's just not going to happen with our current LLMs, they aren't going to scale up to that extend. There are already articles suggesting that we're almost at the peak of what they can do, with the resources we allocate.