How do projects like this deal with papers published based on falsified data? Do they reproduce any of the source data themselves?
How do projects like this deal with papers published based on falsified data? Do they reproduce any of the source data themselves?
I can't speak to this specific instance, but Materials Project does try to pay close attention to questions of reproducibility and provenance. Materials Project runs open-source repos[0] so that its methods can be verified, individual calculations are available via an API[1] and we also partner with NOMAD[2] to make larger files and calculation artifacts available for direct download. This is in addition to documenting methods via peer-reviewed papers, online docs, etc.
This is not to say that issues of reproducibility don't still exist, or that we ourselves couldn't be doing better. It's a big problem in the community.
[0] https://github.com/materialsproject [1] https://api.materialsproject.org/docs [2] https://www.nomad-coe.eu
Btw many people (even trained) fail to reproduce legitimate work. It may not actually be fake or falsified, typically not being able to reproduce an experiment is insufficient evidence for retraction. I didn't watch the YouTube video.
People outside academia seem to believe that reproducibility is an explicit goal when publishing papers. Perhaps it should be, but generally speaking it isn't. Instead a paper is a record of the things the authors found interesting and novel when doing the work.
If they are wrong then you publish a paper with conflicting results. If there is fraud (which is different to "I can't replicate the results") then you deal with the publisher of the paper.
[1]: The importance of stupidity in scientific research | https://news.ycombinator.com/item?id=31977918