I think advocates of sharing science should also understand that, at present, the majority of the content is asinine or incorrect. Removing checks and balances is not helping.
It's not at all clear to me that the checks or balances were helping beforehand. Hiring a shit ton of MBA types to administer research probably didn't help much either.
Also, out of curiosity, is this open access / fast review problem what led to machine learning falling into irrelevance?
Because we wouldn't want that to happen to other fields. Can you imagine if raw clinical trial data was out there, for example? (The Horror! Best keep these appallingly expensive results locked up!)
Why? As the first author of several publications, more than half the reviews I received were not well thought out. This part of the process doesn't happen, Open Acess and self publishing websites exploited this fact for greater throughput.
>>machine learning
I would argue that the machine learning bloom in recent years is because of an availability of new technology from industry, supported by industry. This is also why they were able to tolerate less conventional venues for publication - as these were not viewed as mile markers for progress. Yet, when I look at more "settled" machine learning fields like speech, I see the same plethora of academic papers filled with useless crap.
Perhaps the growth of ML has nothing to do with Open Acess. There is although certainly there is an underlying profit motive for successful technologies, Google ain't going to use a technique that doesn't work.
>>raw clinical trial data
It would be great if it were published, and indeed many people would pay for quality trial data. I see OA and self-publishing of research results having no relation to authors withholding clinical data.
In the end it's the people who may or may not try to build on your work (or mine!) that are the best judges of whether it's BS.
Re: 2: probably, but diffusion of ideas benefits massively from readily accessible documentation. One of the things that will cause me to immediately reject a paper is if its method implementation does not work as described. I will always reject in such cases. They are alarmingly common and this is one of the few easy cases in reviewing. Lack of an implementation is also cause to reject.
Re: 3: it has everything to do with it. Unless you're running and publishing trials please don't lecture me on this. See opentrials for a particularly frightening take on why published trials so poorly represent those submitted to the FDA or registered with CTEP. Meanwhile actual RCTs get buried by shitbags demanding more subjects than there are recorded cases in the past 40 years (no joke).
If you're not doing trials, you may have to take my word for it (or get involved in RCTs and find out for yourself). A lot of the enterprise is actively harmful to science by omission.
"If nobody cared about your opinion, it would not be financially viable to publish it. But now we have blogs/tweets."
Effectively, cost of publishing was working as a quality filter. Now, we need to actually implement proper filters instead of bad proxies for quality.
If I just leave that out you seem to imply science publications should be selected by whether enough people care about, or whether they are boring or not. Both (especially the latter) are rather bad measuring sticks for the quality and importance of research.
Adding "financially viable" to "less boring" produces "EXCITING!", adding it to "enough people care" yields "POPULAR!". These are definitely not forces that should be pulling scientific research.
Not saying there shouldn't be checks. Just that market forces are probably too stupid for it.
Although that's not giving BuzzFeed enough credit, they tend to check their sources & verify results/reports. Also if a piece of fake news is simply resubmitted, they don't pretend like they never checked on it before.
There are plenty of checks (mostly from NIH, mostly for incremental garbage work on dead models like cell lines) and very few balances against the tyranny of CNS...