Hacker News Reader

Comment by yzydserd | Hacker News Reader

> solving problems is only a tool and proxy for achieving the primary goal of conceptual understanding and insight.

This is the effect of AI on most intellectual disciplines, and it’s a real worry.

Yeah. My other favorite example are books. Why do nonfiction books exist? There are some pathologies and corner cases, but fundamentally: to develop and share new ideas. Downstream from that, if it reads well and if you're lucky, you make some money.

But now, LLMs can generate hundreds of books per hour. They make up 80-90% of new arrivals in many nonfiction categories on Amazon. They short-circuit the system, allowing their "authors" to extract money from the system with zero effort by crowding out human work. And it's not even the question of whether these books are good or bad (although overwhelmingly, they're terrible). It's whether it's actually accomplishing anything worthwhile, or just destroying incentives for humans to write or go into any other sort of intellectual work.

In fact, I see many professions push back. Artists, writers, now mathematicians. And I'm amazed that our profession doesn't and that we have so many people who are hooked on vibecoding. I'm still waiting for that 10x payoff. All this velocity and somehow, the landscape of the software I want to use still looks the same as it did in 2021.

I would argue that creating an llm book that sells also requires human labour, just a different kind of one.

It also takes human labor to run an effective nigerian prince scam. That doesn't make it okay.

It doesn't, that's the whole point. You can produce output that passes the smell test with naive buyers in a matter of hours. If you want to write good LLM books, then you gotta work closer to human speed - weeks, months - which allows others to produce 100 slop-books in the same timeframe. You still lose.

Now you piqued my interest. Any good AI-written books out there? I thought it's all slop.

Given the witch hunts around AI creative output, nobody who has put in work to avoid the stigma is going to out themselves.

Why would they have had to put work in?

If an author let AI do 90 to 95% of the work, cleaned up the remaining 5 to 10% themselves, would you say AI or the author wrote it? Because this is roughly the split between an author and an editor, with human writing, now.

The work is either: (1) the act of concealment: generating an AI book that cannot be detected as AI generated; or (2) actually just writing the book yourself.

Playing a devil's advocate. Why do we need understanding ? To take an example i would say ~99% of the population do not understand how combustion engines or how semiconductors work, what say another 1% ?

Is the fear post-apocalyptic in nature ? We need some human priesthood to carry on tradition ? why ?

Let's assume in the next decade GPT-7 PRO Ultra is cheaply ubiquitous, inspectable, reproducible, transferable, reasonably un-constrained by any institutional interests.

What say the 1% ?

Just a thought.

There ought to be more to life than sitting in a pod receiving sufficient nutrients from a tube, even if there was no doubt that humanity would in this way survive until the sun expands and makes the earth uninhabitable.

> There ought to be more to life than sitting in a pod receiving sufficient nutrients from a tube

Even in a hypothetical world where AGI can do everything that humans do better, this would still be a choice to make, not an inevitable consequence for everyone.

And life doesn't require suffering to have meaning.

Huh? Why bring up suffering? I was responding to the question of “Why is it a problem if people don’t understand things?” .

I was saying that human pursuit of understanding of things provides value beyond survival and resources.

Joint pursuit of a common goal is a way for people to better be-in-relation-with one-another.

You can still pursue understanding of things, full automation doesn't preclude that.

What it does is make it a choice rather than a survival necessity. The latter is where the "suffering" comes from.

> Let's assume in the next decade GPT-7 PRO Ultra is cheaply ubiquitous, inspectable, reproducible, transferable, reasonably un-constrained by any institutional interests.

I'd argue that this extremely extreme scenario is the only one in which it kind of makes sense to not have understanding. But let's be honest: no one knows if we'll be there (and it seems unlikely since everything reaches a plateau eventually). So, what happens if we allow ourselves to forget everything and then we don't reach the ideal scenario?

Whole civilizations fell when people forgot how to do important things

I grew up in a cult. Based on my experience, I believe that the most dangerous thing a human can do is to allow someone else to do their thinking for them.

It’s not going to be ubiquitous? There hasn’t been a single frontier model where generation n costs less than generation n-1 to run. So the reasonable thing is to assume that GPT-7 will cost even more than GPT-6, and more and more of the frontier of knowledge will be locked behind a giant paywall. Participating in any field will mean ponying up to the oligarchs that own the infrastructure that runs the model.

Tao basically reiterating "AI is making us dumber" a bit more eloquently and applied to math.

Its by choice. Mathematicians still can use AI to "achieve the primary goal of conceptual understanding and insight"

At the individual level any mathematicians can, but the social group of “mathematicians” will not unless drastic social changes are made, because that's not how the system works today. Drastic social changes usually take decades, and funerals, to occur, so the reasonable hypothesis is that it's not going to change fast enough.

Mathematics belongs to all of us. If mathematicians get depressed about it, I don't give a hoot. Five years of guided computer-aided math could improve billions of lives or solve global warming outright through cheap clean energy.

This is a litmus test for the effective altruists in AI labs: fashion or conviction? The social pressure is to slow down and let the guild keep its norms a few more years. But what's clearly best for humanity is to push forward violently and leave them in the dust. A Fields medalist's feelings are not more important than the wellbeing of the planet.

> Mathematics belongs to all of us. If mathematicians get depressed about it, I don't give a hoot.

They are depressed because what's being destroyed is mathematics, actually.

> Five years of guided computer-aided math could improve billions of lives or solve global warming outright through cheap clean energy.

This is complete BS, math doesn't solve climate change or save life, that's a category error. The purpose of math is to help us build understanding of the mathematical objects and that's it.

Sometimes, after much distillation to the academic world, this understanding is later being leverage by other scientific field, but that's a very rare event when it happens. And it's not how other science progress.

> This is a litmus test for the effective altruists

EA has always been a lie to justify short term individual greed by hypothetical shared future benefits.

Tao has enough moat (name, funding) to make shift happen and even capitalize on this.

youre not wrong, but i think you are missing his base point. tao is saying that mathematics is a practice that the group collectively builds. students and new methods are built on top of what has previously been done, so its a community effort of research, experimentation, and knowledge sharing.

his core argument that is LLMs (and specifically LLMs owned and gated by corporations was my read), while able to solve problems, do not contribute to this practice of knowledge building. solving problems is just one piece, and the mathematics community ingests problems and new methods, iterates and thinks on them, and then produces new ideas, methods, etc. this is what he is defining as progress, and solving things like millenium problems are markers of this progress.

I published a substack about this just a few days ago [1], my core theory here is that we will absolutely have what I call a "highly productive dark age" in mathematics where knowledge vastly outpaces understanding driven by publish-or-perish incentives, but additionally this will lead to the loss of the skills necessary to understand.

The hopeful note is that I do think we are entering a golden age for the curious casual/semi-pro mathematician and for niche mathematics areas that won't get the attention of the top labs. Everyone is sprinting to solve the millennium problems, but this is a very exciting time to be in a sub-sub-field where you and 4 others are keeping things alive.

[1] https://substack.com/home/post/p-214740151

The existence of solutions to problems does not prevent you from solving the problems yourself anyway, if your goal truly is personal development. You're free to go solve Navier-Stokes yourself right now. You're free to manually do all of the AI work in any field, actually. You won't get grant money or prestige (which are not a part of conceptual understanding and insight) but you will get all of the conceptual understanding and insight you're after. You have not been deprived of it.

Except most people that get actually good at a field do so through their day job. By removing mathematician as a viable job, most can't afford to spend the necessary hours to become a good mathematician at all. We're going back to only rich people or those with a rich patron having a chance of developing themselves.

Reply on news.ycombinator.com