Comment by AndrewKemendo | Hacker News Reader

Comment by AndrewKemendo | Hacker News Reader

Parent Full thread

I genuinely don’t see scientific journals and conferences continuing to last in this new world of autonomous agents, at least the same way that they used to be.

As other top level posters have indicated the review portion of this is the limiting factor

unless journal reviewers decide to utilize entirely automated review process, then they’re not gonna be able to keep up with what will increasingly be the most and best research coming out of any lab.

So whoever figures out the automated reviewer that can actually tell fact from fiction, is going to win this game.

I expect over the longest period, that’s probably not going to be throwing more humans at the problem, but agreeing on some kind of constraint around autonomous reviewers.

If not that then labs will also produce products and science will stop being in public and the only artifacts will be whatever is produced in the market

"So whoever figures out the automated reviewer that can actually tell fact from fiction, is going to win this game."

Errr sure. Sounds easy when you write it down. I highly doubt such a thing will ever exist.

Who said it was easy?

If you think these types of tools are going to be generating "the most and best research coming out of any lab", then I have to assume you aren't actively doing any sort of research.

LLMs are undeniably great for interactive discussion with content IF you actually are up-to-date with the historical context of a field, the current "state-of-the-art", and have, at least, a subjective opinion on the likely trajectories for future experimentation and innovation.

But, agents, at best, will just regurgitate ideas and experiments that have already been performed (by sampling from a model trained on most existing research literature), and, at worst, inundate the literature with slop that lacks relevant context, and, as a negative to LLMs, pollute future training data. As of now, I am leaning towards "worst" case.

And, just to help with the facts, your last comment is unfortunately quite inaccurate. Science is one of the best government investments. For every $1.00 dollar given to the NIH in the US, $2.56 of economic activity is estimated to be generated. Plus, science isn't merely a public venture. The large tech labs have huge R&D because the output from research can lead to exponential returns on investment.

" then I have to assume you aren't actively doing any sort of research."

I would wager hes not - he seems to post with a lot of bluster and links to some paper he wrote (that nobody cares about).

Reply on news.ycombinator.com