Between 2016 and 2021 the share of ML/ robotics/ AI researchers being reproducible (ie contianing code and similar instructions to reproduce) doubled [1].
The major US labs have gone largely closed source (I.e. they no longer publish frontier research) but the Chinese ecosystem has incredibly reproducible code.
This is field dependent obviously but I think it atleast gives reason to be optimistic.
Yes people will churn out fake slop research, but it feels like that can be categorized and then ignored.
> if AI can reproduce such research it would be capable of doing such research itself
Well there is a big distinction between research validation and research generation, it is generally much easier to verify that a math proof is true or false than to find a truly novel proof.
But yes in the long run I’d think AI will be doing tons of research and it will by default reproducible. So maybe we’re aligned after all?