> I feel like we need a new way to do peer review, that is more real time - so that papers can be upvoted/downvoted, flaws can be pointed out - and we have some way to assess the truthiness of what the paper is claiming. Your comment is a step in this direction (but we're not capturing the wisdom of the crowds quantitatively around papers today - arxiv is great, but 1990's era web design).
> This one managed to get 700 upvotes on HN and still be meaningless.
Agreed. This is why I don't think voting with online peramters, like down voting, will ever be viable choice; because aside from things like Cybil attacks, the reality is that Social Media has normalized and acutely optimized group think and the tribalism that often follow. For those of that don't engage in it its incredibly alarming and disconcerting how many succumb to it's practices in real Life.
What I will propose as an alternative is something that was first tried with Andreas Antonpolus' first book 'Mastering Bitcoin' wherein the chapters are each individually uploaded to Github and follows the same process that OSS does, wherein commits and corrections are submitted to alter the books ultimate maintained version, and never really being fully 'finished' and can be amended and annotated as needed to suit the updates that follow (eg: Segwit or Lightening Network) or in this case perhaps a replicated experiment that provides a larger sample size. Replication in Academic Papers is nearly non-existant, despite the notion that Peer Reviewed (especially in STEM) was to be a the critical component that made it invaluable.
This could also effectively undo the walled-garden-extrotionist model that afflicts Academia's Peer Reviewed model and foster more interactive and International work without being present in a University. Samples or specimens may need to be more tightly controlled and transported, but this could effectively be done over night if the Will exists for little to no money in anything other than training.
If the International University model is undergoing mass disruption and shifting toward a mainly online learning platform, this too could help mitigate these glaring problems and have a Global Repository for ongoing research.
Its almost stupid not to do it at this point given the MANY pitfalls that its current model forces down our throats.