chongli

I didn't say anything about low-value output. No one actually knows the value of any particular piece of mathematics within that deluge. Mathematicians don't have a magical ability to differentiate high-value mathematics from low-value merely by reading paper titles and abstracts.

The dirty secret in the mathematical world -- that has been going on for a long time already -- is that papers get attention based on the reputation of the authors, not on the rigour or validity of the proof. The big headline-grabbing papers are getting read by mathematicians because AI researchers have leveraged media exposure to bypass the reputation network, but media exposure doesn't scale.

When everyone is using LLMs to generate proofs, only reputable mathematicians will be able to get their work read. And herein lies the crux of the problem: an exponential takeoff in the volume of output from respected mathematicians will leave a critical shortage of readers.

it could end up being far easier to reasonably direct and evaluate the research direction and output of AI systems than human mathematicians

That's baseless speculation. All indications so far are that LLMs produce proofs far longer and far more complicated than humans are capable of, such that only machines can check the proofs for validity. Digesting them into a human-readable interpretation of the results is an open problem.

The issue is SNR: signal to noise ratio. Generating exponentially more mathematics, particularly if the process is indiscriminate or optimized for something other than usefulness or mathematical relevance (such as optimizing for machine-provability), does not imply that we get exponentially more applications. We may end up halting the progress of applications altogether as the entire capacity of the world's mathematical apparatus is consumed by the interpretation and investigation of machine-generated proofs.

You can already visit arXiv and find vast numbers of not-yet-published mathematical papers. Most should never be published. None of this junk is benefitting humanity in the slightest.

That's a misconception. Only a tiny percentage of mathematics has seen any applications whatsoever. There are vast libraries full of mathematics no one (in this discussion, anyway) has ever heard of that no one reads anymore and has never been applied to anything.

This idea of trying to "prove all the math" with AI makes as much sense to me as using chess engines to try to "solve chess."

The area of a triangle doesn't have 1 unique formula, it has many, depending on the system you use. A triangle in plane geometry has a different area than a triangle in spherical geometry, and different again in hyperbolic geometry.

When you get to studying the geometry of manifolds, you realize the area of a triangle can be any damn thing you want, depending on how you construct the manifold you embed it in.

Ultimately, an option becomes a requirement and then the protection for users is totally gone.

Apple on the other hand offers a locked down product that many people want because they don’t trust themselves. I think a lot of people have short memories for all the hacking and spyware that plagued Windows users a couple decades ago. I think we also need to remember that Apple is not the government so people have alternatives to buying their devices.

Personally, I think people should have the right to buy a locked down device. It’s not on Apple to provide open competitors to their product. And why should developers have a right to develop for the platform? Heck, I wanted the iPhone back when it didn’t even have 3rd party apps at all.

I don’t use Windows on any personal machines, so there’s no way for this to benefit me. I hope it helps others who are frustrated by this though!

A physical notebook is so easily lost I wouldn’t even consider it.

Hobbyist mathematics may long outlive a lot of research mathematics, and I think that’s okay. There’s nothing in the human spirit of inquiry that necessitates academia as an all-powerful institution. Heck, there have been a lot of problems with the institution in recent years, and those have everything to do with perverse incentives and nothing to do with AI.

Do you use the same password on every site? How do you deal with data breaches?

If the collar can be deactivated to be removed for servicing/off duty/retirement/replacement/etc then it can be hacked and replaced with a placebo.

That is not true. In academia you have extremely niche research specializations and tenure as two major contributors to inertia. In general software development you have an enormous amount of career mobility.

Software developers learn new languages, new frameworks, new tools, new development methodologies all the time. A string theorist who decides to pivot away from string theory essentially has to start their career over from scratch. It’s not even close!

I can see there is no further productive discussion to be had with you at all. I bid you good day.

[1] https://www.merriam-webster.com/dictionary/incomprehensible

I believe Terry Tao when he says the bottleneck will no longer be the writing of proofs, it’ll be everything else: reading them, reviewing, publishing, and teaching from them. A bunch of proofs that nobody reads are of no use to anyone.

Yes, because I already knew what schedulers are, what spinlocks are, and if I want to know what futex is I can go look it up. Comprehension is within my grasp.

Your original claim, which you’ve repeatedly distanced yourself from (by trying to use comprehensible examples) but won’t admit to, was about incomprehensible stuff. That is, text that no human could possibly understand, ever.

You’re repeatedly engaging in intellectual dishonesty rather than simply admit that “human comprehension probably will continue for the foreseeable future”, which is really not a controversial idea at all.

The age of humans comprehending things is coming to an end [1]

[1] https://news.ycombinator.com/item?id=49312914

Nowhere did I suggest that “the reason for discrediting string theorists is that they have financial stakes in the idea.” Please try to be more charitable than that.

His original claim amounts to creating an AI that takes its place above humans as some kind of electronic God, delivering edicts to humanity that we cannot comprehend, but which somehow have value to us. It's unskeptical, pseudo-religious nonsense.

We already have countless examples of such filling up the arXiv, written by hacks long before LLMs started writing proofs. No one cares about them. You might as well build a box blasting radio static into the void. You could save a lot of electricity that way.

an LLM might e.g. break some cryptographic algorithm in a way utterly unintelligible to humans, but the fact that it works would be sufficient on its own to make all of us choose to abandon that algorithm and choose different ones.

No, that is the entire point. If it is an algorithm which accomplishes something useful, then it is intelligible as such. That which is incomprehensible cannot be understood even in part, so it provides no value as a bit of knowledge (unless you're looking for a strong random number source, I suppose).

There is no value in an undeciphered document until understanding is achieved, just as a lode of gold ore in some asteroid orbiting a distant star has no value until we can fly there and extract it.

If an LLM can help us understanding something then it was not incomprehensible, by definition.