Hacker News Reader

> I've been wondering whether AI really is improving rapidly at open problems or we're being fooled.

I think your suspicions are warranted and your explanation seems plausible.

If better training data is the reason here, it would still be a case of the models doing something that is in and of itself super useful! The models really can take that data and distill it into solutions for similar problems faster than humans can. This is great!

But there's so much vested interest in the AI companies to be opaque about all this, to hype up their models and avoid giving credit to people whose data made everything possible, that they would never tell us this fact if it were true.

I feel like so much of the AI hype cycle is like this. The models develop extremely useful capabilities, but it's hard to understand what they really are through the hype. The lies and obfuscation by their owners who have vested interests in capturing the value they provide makes it impossible to take anything they say at face value.

>> If better training data is the reason here, it would still be a case of the models doing something that is in and of itself super useful! The models really can take that data and distill it into solutions for similar problems faster than humans can. This is great!

It's perhaps great in the short term although it's not very clear who it's great for. I'm not sure mathematicians find it all so great, I mean.

In the long term, if this contrives to destroy the tradition of human mathematics the whole endeavour is self-defeating. In time, there will be nobody left with the knowledge and skills to produce mathematics to train AI to do mathematics.

And then we'll be left with no mathematics at all: we'll have no human mathematicians and no AI that can do mathematics, either.

I was pretty depressed when I read about what happened with Navier Stokes this morning. The Clay Math prizes were a significant motivation through my math career, and I know a lot of computer scientists and physicists that feel similarly. I didn't think I was gonna resolve P vs NP or the BSD conjecture, but I did really research that felt like I was working towards something incredible. What is the younger generation left with? Hey kids betcha can't resolve the Collatz conjecture, our superintelligence can't either! Still pretty depressed about it, to be honest. Intellectualism is dead. We can return to happy agrarianism, I guess. At least the AI doesn't wanna eat my snap peas.

I don’t think this will happen, but it’s possible for humans to adjust our philosophy of mathematical work so that we deprioritize “egotistical” (this is a bad word for what I’m going for, but I mean the desire and economic necessity to associate novel work to your name) discovery and prioritize learning; I’ve never really learned something well without lots of personal insights along the way.

If this is not possible it does make me question whether mathematics ever had any value except for economic or industrial reasons. I do believe it does however, so it must be possible.

This is no time to despair. It's the time to take a stand. If you don't want to see your discipline go the way of the dodo, then do something about it.

I don't know what you should do because I'm not a mathematician. But superintelligence schmuperintelligence. We didn't stop running because we have cars or playing chess or Go because there's chess and Go engines. Even more so than chess there's no point in maths unless it's people doing it, for other people. AI maths makes no sense, like AI art makes no sense, because those are things that people enjoy and can do pretty damn well ourselves so there's no point to automate them away. We gotta stop that bullshit, and we can stop it. And if we don't, if we just sit around and wait for OpenAI and Anthropic to destroy society then that's not their fault but ours.

Sorry, I'm not great at pep talks. Those are brave men. Let's go kill them!

> AI maths makes no sense, like AI art makes no sense, because those are things that people enjoy and can do pretty damn well ourselves so there's no point to automate them away.

There are people who appreciate and have use for math and art, but don't or can't produce. Those same people are now given tools to produce works that they enjoy and can do pretty damn well at.

> Those are brave men. Let's go kill them!

This is a horrible perversion of the quote. Aside from it's questionable use in this context (as it can be construed as a violent call to action), Tyrion made the battle cry for self-defense against a threat to their lives. Generally, AI reducing the dependence of one human on another's work is incomparable to this.

>> Those same people are now given tools to produce works that they enjoy and can do pretty damn well at.

It's the tools that produce the works, not the humans.

>> This is a horrible perversion of the quote.

What? The full quote is "Those are brave men knocking at our doors. Let's go kill them!". What's the perversion?

A mental model I was thinking about was - I remember when Travis Kalanick was talking about using the chatbot to discuss “vibe physics-ing” on the all-in podcast.

And like - I think there’s a presumption you could make that AI models could overfit to asymptote towards just the capabilities and knowledge we currently have.

And that would be amazing! And crazy useful. And there are probably a whole world of complex problems that remain unsolved because they’re adjacent to knowledge we have but they haven’t been invested in.

But can a human reliably tell the difference between “can do 99.999% of the things we currently know how to do which includes a small subset of things we didn’t know we had the capacity to do” and “super intelligent math and science research pushing the frontier of what we know”

A physicist that knows all the things we currently know in excruciating detail feels like it should be able to make the leap beyond the frontier.

But since these are computer models it might just be that it can ride that line extraordinarily well while the line remains firm.

> - OpenAI invites researchers to use their models, in fact giving at least 100,000 researchers free access[1], but there are also those that pay

I've been thinking along exactly these lines... they very well could have a 21st century Mechanical Turk and its real superpower is getting people to "collaborate" asynchronously but it's just stealing their ideas and laundering them.

I don't think it's purely that, of course... but "consult other clients' transcripts" would be an easy tool to write.

there are also attempts to crowdsource human research directions - like the caltech mathathon challenge : https://mathathonchallenge.com these would help models on the same problems at the expense of the researchers. basically, math researchers are the reverse centaurs but they dont realize it.

There is a very active open letter of over 1000 signatures from mathematicians in protest of this event. This event is targeting undergraduates. It previously suggested that math researchers already have no place in mathematics, and presents a limited and heavily distorted view of what mathematics research is.

I'm an AI skeptic, but I don't see how this squares with what the organisers of the event actually say. "It previously suggested that math researchers already have no place in mathematics"? I don't see this.

The website previously said, “What is the role of a mathematician when AI can solve conjectures faster?” but they have removed it, possibly as a result of the letter since it happened after.

Also I want to mention that the letter is not about AI skepticism, in any direct way at least.

>> Internal OpenAI models are reportedly solving open problems at a surprisingly fast rate[2]

Maybe I'm failing to read that graph properly but the y axis says "pass rate" and it only goes up to 0.5. That would mean every single problem is at most half-solved.

I don't know what that means though. What is "0.5 pass rate" in the context of "open math problems" (as in the graph title)?

I guess it's a fraction of problems on which a model produces a LEAN proof or a counterexample.

Wouldn't they just list the number of problems solved then?

rates beat counts almost always.

You're assuming too much from a single graph with vaguely named axes.

I'll wait until there's more concrete information.

Can't both be true?

1. Systems that OpenAI is able to use (either public or private) are improving rapidly at open problems, even if they are still extraordinarily expensive

2. Researchers will inadvertently speed up the rate at which the AIs improve by feeding them valuable training data

This is pretty much the definition of a data flywheel.

It tells me that AI companies are just another mechanism to extract and extort value from the masses for the rich.

Just another rich man’s trick

Perhaps the last one before they destroy that world and try to hide away as people forget and history is rewritten again. I don’t think they’ll succeed this time.

AI providers are pretty much the end boss of rent seeking, that’s for sure

The relentless progress towards saturation of benchmarks is, I suspect, at least partly a similar story. Whatever holdout questions are used to evaluate GPT-x will be in the GPT-x conversation logs, and therefore in the training set for GPT-x+1.

I can't think of better invention than one that can saturate a benchmark of every interesting problem.

Seems easy to picture high stakes startup cutting corners to justify their fame.

> they could be significantly piggybacking on human progress,

This is AI in a nutshell, its a plagiarism machine. An abstraction layer between vast amounts of stolen human-generated data that filters out the liabilities and accountability for that original theft. Its an IP laundering system.

That’s one perspective.

I just view it as a thing that can brute force and produce outputs - that it has no way of ‘knowing’ - but doesn’t need to since it’s just running off of probability.

No human can compete in that contest. But no llm can compete in the contest of ‘understanding’ and application in the real world - which is where 99% of the value is.

I’m very pro AI long term btw but I’m not blinded.

But it’s not brute force if it’s looking over everyone’s shoulder

Brute force would have been solving Navier-Stokes in 88 hours after plagiarizing all known 20th century math

When it needs to snoop live on what the actual mathematicians are working on that’s something else

No its happening whilst the human is working with it. The new inputs provided become part of the brute-force. This is what Scam Altman means by 'self-recursive'.

Trust me I've seen it happen to myself. I no longer trust ChatGPT.

I can see right through his act. Altman is one devious f8k.

AI needs humans to encode ideas in words. It needs those ideas to span the space of possibilities of, say, Navier Stokes. Then AI can be, as you say, a terrifyingly effective way to search that space.

But when the building-block ideas are still being formed, I'm not sure that AI is good at forming them.

COrrect and this is how labour displacement happens.

There are many actions being performed today that can be nicely packaged.

Im already working on such a project.

Dont forget the holisitic validators/tools in the process. Probabilistics alone likely will not get you here. These rules are human made and without it, frontier models would not be able to compete, likely.

That's such an unquantifiable accusation.

Plus it is an unfair standard since so many scientists in the past have been caught unethically using the work of others without attribution (and so many more have been accused).

In history we also repeatedly see the phenomenon of multiple discovery or simultaneous invention. If that happens to AI because the topic is pregnant, would you call it "plagiarism" just to disparage AI? https://en.wikipedia.org/wiki/Multiple_discovery

Your first example is the apt one here. In this case openAI was, allegedly, pilfering the work of the scientists into the AI.

How is it an unfair standard. OpenAI stole the work of others to build the AI. That's not different than scientists stealing from other works as their own, or artists copying others work as their own, etc. It's all plagarism. I'm applying the same standard for everybody.

As for multiple discovery, this is a thing, but I don't think the AI did a parallel discovery any more than Ray Kroc made the parallel discovery of the MacDonald brother's speedee service system.

I’d argue the invitation of researchers was incredibly strategic.

Sam Altman knows what he’s doing. He will happily screw these folks to one-up his competition.

The pudding is in the proof. The field is mathematics, the proof can be rigorously verified. If there is a flaw, OpenAI is out to lunch. If the proof is valid, OpenAI has produced something new.

Did you read what they said? The question is now if OAI produced something new or just stole the researchers' good ideas.

Name one discovery ever that didn't depend on someone else's work.

Most discoveries did not happen by someone looking in someone else's notebooks without them knowing.

You seem to be unfamiliar about how research works. It's common to make an incremental advancement while citing prior work. The vast majority of papers out there fall into this bucket. Did the AI make incremental progress? Yes. Did it cite prior art? After some nudging, yes.

It seems to me the academics are upset that AI scooped them. But scooping is a time-honored tradition between researchers. First to print and all that. In a nutshell, they are upset that they lost out on a publication.

I will also point out for those unaware that any mathematics that is produced is automatically part of the public domain and can be used freely in derivative works. It is not a protected intellectual class like other works of art.

> But scooping is a time-honored tradition between researchers.

Provided that it's properly accredited. And definitely not for others' unpublished work -- that's despised upon if not an academic integrity issue.

People even point out that you should add a reference to certain papers during the peer review process.

Scientific papers many times have citations of the kind "private communication." APA has a style guideline so certainly not looked down on: https://apastyle.apa.org/style-grammar-guidelines/citations/...