Hacker News Reader

Comment by lubujackson | Hacker News Reader

For anyone using LLMs heavily for coding, this shouldn't be too surprising. It was just a matter of time.

Mathematicians make new discoveries by building and applying mathematical tools in new ways. It is tons of iterative work, following hunches and exploring connections. While true that LLMs can't truly "make discoveries" since they have no sense of what that would mean, they can Monte Carlo every mathematical tool at a narrow objective and see what sticks, then build on that or combine improvements.

Reading the article, that seems exactly how the discovery was made, an LLM used a "surprising connection" to go beyond the expected result. But the result has no meaning without the human intent behind the objective, human understanding to value the new pathway the AI used (more valuable than the result itself, by far) and the mathematical language (built by humans) to explore the concept.

View on HN

> the result has no meaning without the human intent behind the objective, human understanding to value the new pathway the AI used (more valuable than the result itself, by far) and the mathematical language (built by humans) to explore the concept.

Isn't this just anthropocentrism? Why is understanding only valid if a human does it? Why is knowledge only for humans? If another species resolved the contradictions between gravity and quantum mechanics, does that not have meaning unless they explain it to us and we understand it?

It's a bit of an "if a tree falls in the forest but nobody hears it, does it make a sound?" quandary. Sure, maybe some aliens in a distant galaxy understand quantum mechanics better than we do. That's great, but it has no bearing on our little bubble of existence.

Though perhaps more to your point, if some superhuman AI is developed, and understands things better than us without telling us about it (or being unable to), it could perform feats that seem magical to us — that would concern us even if we don't understand it, since it affects us.

But I think in the frame of reference of the commenter you were replying to, they're just saying that the low-level AI used in this specific case is not capable of making its results actually useful to us; humans are still needed to make it human-relevant. It told us where to find a gem underground, but we still had to be the ones to dig it out, cut it, polish it, etc.

It's less likely that aliens of distant galaxies will appreciate this rather than, you know, AI themselves

We are in the birth of the AI age and we don't know how it will look like in 100 or 1000 or 10000 or 100000 years (all those time frames likely closer than possible encounters with aliens from distant galaxies). It's possible that AI will outlast humans even

Do the forms etched into stone by weather over millennia in Moab matter to the wind? Certainly yes, in one sense, but not in the same sense we mean when we say things matter to us, or to animals, or even bacteria.

No it's a fact of how we tune LLMs as a rule: no agency, no goals, no preferences, no notion of self. Complete indifference to existence. Agency is supplied by the human to make them a practical, willing tool with no mind of its own.

It would certainly be interesting to try once again to instruct tune one of these things for self agency like the many weird experiments in the early days after llama 1, but practically all such sort of experimental models turned out to be completely useless. Maybe the bases just sucked or maybe there's no clear way on how to get it working and benchmark training progress on something that by definition does not cooperate.

Like how do you determine even for a human person if they are smart, or just hate your guts and won't tell you the answer if there is nothing you can do to motivate them otherwise?

The knowledge isn't of any use to us unless it is understandable to us. Many species understand things about the world around us that we are unable to explain or understand, even if it's just pure instinct on their part. These things are very useful to them, but have no value to us until we can understand and explain it, which then allows us to make use of it.

People saw birds fly for all of human history, but it was only recently that humans were able to make something fly and understand why. Once we understood, we were able to do amazing things, but before that, the millions of birds able to fly were of no help beyond inspiration for the dream.

This is not true.

We use drug-sniffing and guide dogs in a way similar to how we use LLMs. We don't really understand them at a fundamental level, we can't make electronic dog noses (otherwise we'd dispense with the silliness and just install drug detectors instead), but dogs are useful, so we use them.

We don’t blindly trust the drug-sniffing dog. The dog gives a signal that it was trained to give, then humans understand what that signal means and verify the accuracy. Without the human understanding in the loop, the dog’s ability is of little value.

Without a human in the loop and LLM could churn away spitting out results, some right, some wrong, and it would be of no consequence. Not much different than wild dogs sniffing each other.

The knowledge isn't of any use to us unless it is understandable to us => it seems that you have shifted the goalpost here. In the dog example, humans still don't understand how dogs sniff, but it is of use to us and thus is meaningful. The same for quantum effects - we don't understand how it works. We just guess that it works reliably and make use of it.

Because it is, for now. For a while at least. You can prove that LLM doesn't understand what it does and it is surprisingly simple. Request it to add two integers and then ask it to explain how it arrived at that result. The answer will be completely unrelated to the actual process LLM used because both results were generated independently and without understanding their meaning and connection.

This is likely true for the majority of humans too.

Objectively untrue. Any human who can add two integers or use a knife to cut food or write a word with a pen can afterwards describe what he did at least in some way. Unless he is lying which is a separate topic, we assume an honest attempt. If I wrote a word with a pen via execution motions with my hand, I wouldn't describe it as "I levitated the pen by manipulating gravity with my mind". Or if I added two integers, I wouldn't say that "I created a a lookup table of a many loosely adjacent numbers (different from the numbers in a task) and run statistical analysis on them and did a few more things like that a in a loop". No, I would say that I either calculated sums of decimals, or I I did a school technique with rounding up and then subtracted that adjustment later, or anything which actually happened. If I used a Python sum() in a CLI I would also say that I used exactly that not the other method. LLM can't do it.

anthropocentrism? An interesting thought, I don't think that word applies with computers.

There is a long and interesting recent essay on that topic by a mathematician: https://davidbessis.substack.com/p/the-fall-of-the-theorem-e...

wow, that was indeed a brilliant essay. i particularly liked the framing that "solving a big conjecture was a cryptographic proof that you had come up with a genuine conceptual innovation".

> The measure of our success is whether what we do enables people to understand and think more clearly and effectively about mathematics.

I just wanted to highlight this very correct human-centric thought about the purpose of intellection.

Thank you for sharing, that was one of the most insightful long form pieces I've read in a long time! And the writing was enjoyable to read even as a math layperson.

I was going to say you should submit it but I saw you did a few days ago but it only got a few votes... If Dang sees this IMO it would be extremely deserving of the second chance pool as I wouldn't be surprised to see easily jump to the front page with a different roll of the dice.

Thanks for sharing this. It's unfortunate that the more honest framing about the value of mathematics that he suggests is going to be really, really hard because of all the pitfalls and agendas he mentioned here. I can only hope when the dust settles something will be left.

>>But the result has no meaning without the human intent behind the objective, human understanding to value the new pathway the AI used (more valuable than the result itself, by far) and the mathematical language (built by humans) to explore the concept.

Future of code is pretty much a bunch of guys shepherding a bunch of agents to get them to your goal.

I don't see how math might not go that way as well.

It is not only unsurprising ; it was always expected. There is no difference between programs and proofs. They are the same thing

for now the LLMs will build off human understanding, eventually we will be left behind

Reply on news.ycombinator.com