In fact, the best place to find this information is through the primary literature. You can use Google Scholar and Sci-hub to do your own research. For most issues, it's actually not to hard to understand the primary literature, and it will be much more accurate than the crap you read on popular websites.
And then if you point out a problem with a research undertaking or study when it is brought up, you can get some set of true-science-believers looking at you like it can't possibly be. It's unreal sometimes.
It's really no wonder people spread their subjective experiences as if they are science. Institutional "science" is great as a set of models and perspectives, but when you dip into the product you can start to wonder if you are effectively doing the equivalent of product research by watching the home shopping network.
It's scientism pure and simple.
Or to quote a bunch of people who had a great idea: Nullius In Verba.
"Here’s a dirty little science secret: If you measure a large number of things about a small number of people, you are almost guaranteed to get a “statistically significant” result."
"A woman’s weight can fluctuate as much as 5 pounds over the course of her menstrual cycle"
As a regular person how would I know what the correct sample size for this type of study should be? Remember, that number isn't some fixed value but a range that increases confidence (if I'm using that wrong sorry) and it's all subjective.
I would also have to think about the uncontrolled factors, like mensuration, as having an effect since he didn't mention it in the original study.
This is far too complicated for the majority of people.
I didn't even think of this. You could be looking at a study that turned out to be disproven or even fraudulent (though maybe those are removed from journals). This is why there are experts in particular fields who keep up to date and should be the source.
The problem is the current wave of anti-intellectualism which turns everything into black and white easy to digest situation. A person lies once, don't trust them. A newspaper printed one or more articles that turned out to false, don't trust them. Doesn't matter how long ago or how many other truthful articles, they are done. However, like the GP tried, they don't offer any reasonable solutions to how are you suppose to get reliable information.
I wonder if the true goal of some people who push equal mistrust of historically standard sources of information is to make it easier for people to lie.
The number of times I was told that I should shut up about covid because I'm not a epidemiologist was astonishing. What's more I was criticizing their _computer models_ something I do have a degree in.
If anyone asks you if you "believe in climate change" then you're talking to someone who does not understand science and instead has fallen for scientism. The actual question is "do you understand climate change" since belief is not required.
I blindly believe in almost everything in science. For example gravity, I don't understand gravity beyond some of its effects.
You think that's strange? Should I say I don't believe in gravity? Belief and understanding are two different things.
Let me pivot here and say "in most situations should just believe whatever the majority of experts in a particular field say is true but that has the highest probability of being true"
Because it has the highest probability if being true
https://www.catholic.com/magazine/online-edition/science-is-...
But 30 is a bare minimum. 60 is better (ie 30 active, 30 control) and the ideal is in the thousands.
Even the fact of taking money to be in a study is already non-normative, and has to be controlled for.
So, take any study on human behaviour with a huge pinch of salt.
That said, some studies can be replicated, and correlate well with what we experience. Dunning-kruger and loss aversion are easily (and often) replicated, but also correlate well with everyday experiences.
When it comes to diet and weight-loss though, well, consider it all rubbish IMO - 99% is rubbish, and figuring out the 1% is hard.
My favourite example is if you have a pill you think allows people to fly and you administer it to 1 random person and they shoot off into the sky? Despite sample size 1 that's actually pretty strong evidence :D
Or they could just ignore any study with p>0.005 (extra zero avoids a lot of p hacking) and suffer no adverse effects.
The latter is probably a better rule of thumb to give the masses.
I largely disagree with the term p-hacking as its not specific enough to the nature of the violation (omitting experiments, straight up [partial] fabrication of data, ...). A problem is that often it is odorless. What is the odor of omitted data, omitted experimental runs, ...?
Regarding "data dredging" i.e. testing many candidate correlations, I have mixed feelings. As long as all tests are mentioned or at least the number of tests provided I don't see a problem really.
A stream of 100 papers stemming from a 1000 different studies each truthfully based on a single test excluding the null hypothesis over its dataset at p=0.05 and a single study testing 1000 properties and finding 100 at p=0.05 have the same expected number of spurious and real findings. A scientist with more programming skills or support, more compute, ... will simply find similar quality results faster. Scientist B may intuitively feel jealous about the quantity of similar quality results that scientist A picks up with a dragnet approach, and the jealousy may or may not be justified (not having the same amount of compute, support, training, etc.), but that does not justify accusing scientist A of fraud, in those cases where there is none. Just like one factory worker might work faster than another (after part of his job on the factory line became automated) that doesn't mean its being done less well.
What disturbs me the most about the article is the following:
>Luckily, scientists are getting wise to these problems. Some journals are trying to phase out p value significance testing altogether to nudge scientists into better habits.
This worries me enormously. Plenty of non-scholarly articles, like news articles mess up units, especially when time is involved. Product information also often contains rather intentional unit confusion. I hope the solution wouldn't be to simply ban units!!!
If some people use erroneous arithmetic, should we collectively ban numbers too?!
In the face of malpractice we shouldn't do away with the theory, we should think of ways to force practice to adhere to theory.
Again, I fully agree with your suggestion to set tighter significance levels. When we observe readership complaining about the huge number of results, and of its low trustworthiness, the most obvious solution is to set stricter significance levels. At some point levels will become tight enough that explaining outright fraud will become so hard because the probability of an effect later discovered to be nonexistent resulting in a spurious false positive at the stated p-value becomes embarrassingly low.
If the dataset sizes are chosen large enough, having separate test and validation sets can help tremendously.
I started drinking coffee because I had heard that it was the primary source of antioxidants in the U.S., and I decided to start adding unsweetened cocoa powder to every cup.
I get all the health benefits of chocolate this way, but none of the fat or the sugar.
The taste is not bad.