Hacker News Reader

Comment by gowld | Hacker News Reader

How is that different from traditional research publication? It's faster?

Top line difference is speed. The impact of speed is deeper and gets felt over time.

Traditional research publications are no angels. They gatekeep research, and also allow financial incentives to drive them to publish junk with their stamp on it.

But a flood of papers doesn’t actually mean more knowledge. In bypassing this route entirely, AI has swiftly lost the ability to engage with itself as a field. And the cost of that is only beginning to be felt.

That pesky “scientific method” bothers these corps sometimes, so one can play fast and loose with numbers and conclusions for all sort of reasons. No need to worry about reproducibility, rigor, scrutiny, etc. not good for IPO and stocks.

That plus leaning on legacy reputation of journals or schools, instead of open websites and virality.

It's rather similar to publishing a non-peer-reviewed preprint on arXiv.

No traditional research is mostly done by post-docs and have a phd level of education rather than a tech bro that passed leetcode. That seems like a good bar to have; not too mention the whole peer review thing, hard to really understand anything if you purposely withhold it and tell people to kick rocks.

Plenty of research is performed by non PhDs.

The main difference is reproducibility and peer review process.

outside physics/math a lot of published research isn't reproducible at all. There's been several studies that show this.

> There's been several studies that show this.

Were they peer reviewed and reproducable?

And what's the point of claiming there are peer reviewed reproducable studies that show this, if you don't bother citing those studies for your peers to review?

Science is a method of inquiry, not an institution or group of people with titles.

Sure, but it is good that people actually adhere to that method if inquiry.

They are not perfect, but institutions like peer review or universities do provide a framework and incentives for quality.

Eh. Peer review isn't central to science. The point of peer review is: you have a claim someone makes; if it's been peer-reviewed, you can build further research on it without having verify the claim yourself. If you're building technology directly, any result you use you're going to verify inherently, because it either works as advertised or your tech doesn't work. The external verification doesn't add that much in that case.

Peer review isnt central, but it doesn't try to ensure a method is being followed.

If you are spending time, money, and risk based on a claim, verification still helps to cull info that is deceptive or done with methods so shoddy it might as well be lying.

Sure, you can verify through your failures, but that isnt great. You company goes belly up and you can confidently say the blog was bullshit.

Two separate axes here: whether the research is in a blog, and whether it’s peer reviewed.

On the first, I mostly don’t care.

On the second, that’s mostly unavoidable if they want to keep their IP.

Reply on news.ycombinator.com