A few points:
- The basis of a p-value is very much aligned with the scientific process in that you arent trying to prove something 'is true' rather you're trying to prove something false. Rejection of p-values / hypothesis testing is a bit like rejecting the scientific method. I am lucky enough to be friends with one of the physicists that worked on finding the Higgs Boson and he hammered it into my head that their work was to go out of their way to prove the Higgs Boson was a fluke - a statistical anomaly - sheer randomness. This is a very different mentality to trying to prove your new wunder-drug is effective - especially when those pesky confidence intervals get in your way of a promotion or a new grant. Its much easier to say p-values are at fault.
- Underpinning p-values are the underlying distributional assumption that makes up your p-value needs to match that of whatever process you're trying to test else the p-values become less meaningful.
- The 5% threshold is far too low. This means at least 5% of published papers are reporting nonsense and nothing but dumb luck (even if they got lucky with the distribution). If the distributional assumptions arent met then its even higher. Why are we choosing 5% threshold for a process/drug that can have serious side-effects?
- p-value hacking. So many sneaky ways to find significance here. Taleb goes into some detail into the problem of p-values here https://www.youtube.com/watch?v=8qrfSh07rT0 and in a similar vein here https://www.youtube.com/watch?v=D6CxfBMUf1o.
Doing stats well is hard and open to wilful and naive abuse. The solution is not to misuse or throw away these tools but to understand them properly. If you're in research you should think of stats as being part of your education not just a tickbox that is used validate whatever experiment you're doing