Hacker News Reader

Comment by schiffern | Hacker News Reader

  >The advent of AI was always going to be a seismic shock, but this huge dump of papers is a tsunami that we have no time to prepare for. It’s clear that OpenAI will happily wash away our community’s structures to further their financial interests.

What is the alternative?

1. Don't do the research internally (except someone else will do it once the model is public)?

2. Do the research internally but wait longer before telling anyone (this is what OpenAI did before, and mathematicians explicitly told them don't do it)?

3. Don't develop better models at all (except that Chinese models are only a few months behind)?

None of the options available to OpenAI would seem to solve the above concerns.

View on HN

This is a very reductive take; there are so many other options available to OpenAI, many already suggested to them, here are a few of the many other options they have:

Spend the time (as mathematicians do) to cleanup, verify and explain properly any results they have found.

Pay mathematicians to spend their time doing the above with the results they do have.

Improve their LLMs so that they can do the work properly.

Dumping unverified results like this on the community and wasting other people's time with the gibberish their LLMs have produced is the height of hubris, but it leads to a good headline for the IPO.

What’s to stop people from ignoring these altogether and continuing their own work?

“Dumping unverified results” seems to imply that those results come with an obligation of some kind.

Sure mathematicians can do that, and it’s probably the best response to irresponsible behaviour like this, but we’re talking about OpenAI’s behaviour here.

Let mathematicians do the AI-supported research on their own incentive at their own pace? What leaves a bad taste is that OpenAI doesn’t do this because they care about the actual math. Especially when they are leaving the necessary work of properly verifying, understanding, and writing up in an intelligible manner the proofs, up to the math community.

People would be more sympathetic if this was an alien race sharing their math results in not-quite-intelligible-to-us papers, because there at least we could assume that the aliens cared about the math and did their best to transmit their understanding to us.

  >Let mathematicians do the AI-supported research on their own incentive at their own pace?

After they release a model, it only takes a handful of scrappy young math grads (incentivized to publish high-impact papers) to independently put in those exact same prompts, and more. The tsunami remains.

How is it worse if OpenAI hires those same grads internally (vs independently) to run the same prompts?

  > leaving the necessary work... up to the math community

So now it's better if OpenAI hires the mathematicians internally (vs independently) to verify any results before release. I wish people would make up their mind!

Again, that's what the math community told OpenAI they don't want. They said do release any internal results early, rather than withhold them.

Or are you saying.... OpenAI should never run an internal test involving math proofs, for fear of this 'tsunami' of potential discoveries they'd be forced to release? Isn't that a bit like the scholars refusing to look through Galileo's telescope? :-/

Except they don't too the full work. If the prompt also verified that what was proven is what what asked for, and that the proof is correct, and then write it up in an understandable fashion, then the work is done. As it is now it is more like a spoiler for a movie than the actual movie.

For OpenAI/Anthropic/etc it's apparently not enough to be the tool that helped a mathematician with a discovery/proof/etc.

They need to state it was entirely autonomous.

I don’t blame OpenAI for that. There’s a ton of weirdly politically motivated denial of AI being useful for solving problems that require new answers. This is one of the very few possible ways to objectively prove that AI can do that.

What's more, it's quite possibly one of the best ways to improve AI, as math provides both a large corpus of training data and an objective way to measure capability. IMO this is likely why the AI companies are working on this, with improvement in math reasoning bleeding over into improvement in general reasoning.

Really, mathematicians should be thrilled about this, as this could be the biggest practical payoff of thousands of years of investment in math, bigger even than all the preceding science and engineering payoffs. We're told one of the benefits of a math education is learning how to think; was it really so unpredictable if that also applied to machines? This sort of payoff would justify wide open checkbooks for more math research, even by humans.

Well yeah, their ultimate goal is to develop "highly autonomous systems that outperform humans at most economically valuable work"

1) is probably phrased wrongly. I think it isn't even research for the AI labs, it's just an inevitable byproduct of how they train and verify their models. That goes perhaps to the heart of how the mathematicians feel about it: what they thought was a crowning achievement of the human intellect is now a byproduct of a mechanical-industrial process. But that also means the AI labs can't just stop, because they need ever harder puzzles to be part of the training.

I choose option 1. It's in the name! OpenAI should give mathematicians the chance to do at least some of this work themselves. It would be a lot easier for the community to adjust if they could use these same tools to explore how math research can adapt to the new circumstances. Instead, OpenAI is racing ahead to get as many results as possible with a proprietary model that the mathematical community cannot access. If these companies are building AI to benefit humanity as they assure us, then shouldn't they share their resources rather than vacuum up everything for themselves? Especially for something with as little direct monetary value as pure math.

I think there is way too much focus on how they released the results; it was going to be a huge shock no matter what. We should be talking about the massive concentration of "mathematical wealth" occurring here, and how it is a microscosm for what may happen on a much larger scale as AI outperforms humans on more and more domains.

Reply on news.ycombinator.com