Hacker News Reader

Comment by trio8453 | Hacker News Reader

How is withdrawing a paper irresponsible?

it's like if you vibe coded something and the onus is now on the reviewer and the reviewer tells you your work contains bugs and is messy - that is not acceptable from the reviewers pov - why should the reviewer spend all his human effort, a scarce resource, reviewing your code while you've spent barely a fraction of his effort generating this. OpenAI is a trillion dollar company, surely they can verify stuff before pushing it out? The problem is not that they are solving the problems, they don't care at all about the actual process of doing mathematics. Using compute to mine problems and throwing results in github and letting human reviewers spend effort to correct these is not going to win them any favor. If OpenAI really cared about math, they would have someone on their side who actually understood the results they produced and was able to verify their correctness and educate others.

They only withdrew it after someone pointed out that it was a flawed paper.

In one case by asking Astra to review it.

Not exactly encouraging that they did their homework before publishing results.

This is how science is meant to work.

Yes. A researcher who achieves a reputation of making mistakes carries that reputation.

Most good scientists work hard to prove results to themselves before going public.

Peer review is then done _in private_ before publication as a check on quality and significance.

Retracting a paper is pretty embarrassing. And not considered science as usual.

Retraction is after publication and peer review, no?

The greatest proof ever written by man, Andrew Wiles' FLT, was published with a flaw before being retracted and reworked. It's _completely_ normal in mathematics to find flaws in the argument. That's what peer review in mathematics is _for_.

Besides which it's not clear if these papers are considered published or preprint since they appear in no journal, so it's not really a retraction.

It's also very very normal to post preprints on Arxiv before peer review, so it's not the case that mathematics is kept private during review.

Not true. Wiles presented his proof in a series of lectures in 1993. The error was found by a reviewer, and he spent over a year trying to fix it. Once he did fix it, he asked colleagues to review it again, and only once it passed that review did he actually publish it.

No, scientists check their work before releasing it. Slop is still slop if you wrap it up as an academic result.

No, it's not how math works at all. Math is not experimental science. Results take years to formulate and are generally known to be true by colleagues via correspondence and seminars before a paper is even written. Retraction is very rare.

I don't know why people are talking about something like this that they are clearly not familiar with.

Withdrawing it, in itself, might not be, but it kind of skips over the part where they have a paper that warrants it in the first place.

Withdrawing it isn't irresponsible. I don't even think releasing it is, because that is often the best way to find problems with it; expose it to the wider research community. Doing PR victory laps based on unverified, unfinished work, on the other hand, is. It is well known, even scientifically demonstrated, that retractions or refutations get much less attention than the initial exuberant PR announcement. And thus this practice contributes to misinformation.

And just to preempt the kneejerk whataboutism: Yes, all of academia does this to varying extents, and the mainstream media are also complicit. And that is also irresponsible. And no, that is not an excuse for OpenAI. Especially when you consider that OpenAI actively portray themselves as some kind of moral arbiter on AI and "doing good for humanity". They should be held to the extraordinarily high moral standards they purport to hold themselves to.

Reply on news.ycombinator.com