Hacker News Reader

Comment by jjk166 | Hacker News Reader

This is why the goal of experiments needs to be falsifying hypotheses. If you can make a model that fits the raw data without absurd assumptions which contradicts your hypothesis, then the data doesn't support your hypothesis.

If you're checking whether a coin is fair and you toss out significantly more tails than heads because you didn't feel they were proper tosses, of course you're going to reject the null hypothesis that the coin is fair. Even if your criteria for what was a proper toss is objective and reasonable, you're no longer testing whether the coin is fair, you're only testing whether your criteria for counting a toss is fair assuming the coin isn't.

When you construct your experiments to reject a hypothesis - this particular model can not be right if we see this - then you can make real progress towards truth.

View on HN

Then people will simply falsify the logical inverse of their current hypotheses. Preregistration is a more promising approach.

Falsifying the logical inverse is fine. The key though is that showing the inverse of what you believe can not be true is much harder than showing that what you believe might be true.

Preregistration only helps with a select set of researcher bias, specifically where they retroactively change parts of their model to get the intended result. However it does nothing to protect against bias baked into the model ahead of time. Presumably the same reasons that researchers are strongly politically opinionated are the reasons they emphasize certain things over others in their models long before they see any data.

Falsifying the logical inverse of X is identical to verifying X. There's nothing about negation that does anything here. You're making the same mistake people make when claiming "You can't prove a negative".

> The key though is that showing the inverse of X can not be true is much harder than showing that what X might be true.

This is nonsense modal logic. You're saying ¬◻¬X, which if necessity and possibility are duals, is equivalent to ◇X, and otherwise an irrelevant statement. The inverse of X is ¬X. ¬¬X is logically equivalent to X.

> Falsifying the logical inverse of X is identical to verifying X.

That's exactly the point. The issue with positivist hypotheses is that you can find evidence that supports, but does not actually verify, the claim. This seems convincing until you try to flip it around. So for example if a prediction of my model is that the sun will rise tomorrow, and the sun does rise tomorrow, that seems like it supports my model. But if my model can be wrong and the sun would still rise tomorrow, then looking for the sun rise was never going to answer the question.

If you are doing a test that will actually verify X then bias doesn't matter.

I'm not sure what this has to do with positivism.

Data is evidence for all models that predict the data. It doesn't matter that almost all of those models are extremely wrong, while the most accurate models are gross approximations of the truth. Scientists can continue to collect data and narrow the scope of statistically likely models. I don't see how your comment supports your assertion that "the goal of experiments needs to be falsifying hypotheses".

If by verify, you mean prove, I'd say it's not possible to empirically verify anything.

how do you falsify a hypothesis in social sciences?

Same way as the hard sciences - you make a prediction for something that you would never observe if the hypothesis is true, and then you go look for it. If you find it, then the hypothesis must be rejected. I might suspect that hunter gatherer tribes don't go to war, and I might observe many such tribes which don't, but that doesn't prove my hypothesis right. On the other hand, if I can find just one tribe which does go to war, then the hypothesis has been falsified.

The problem is that there are no two equal situations in social sciences, so you won't ever have the same set of initial conditions. I don't know why they call them sciences, but the scientific method is intrinsically incompatible with social phenomena.

You won't ever have the exact same set of initial conditions in any experiment. You drop two balls from the tower of pisa and do it again you have different air conditions, the planet has moved, the tower may have leaned slightly more, etc. You either control for these variables or assume they don't matter. The value of science is that even if you drop two balls on the moon, you're still going to get the same result after controlling for differences - the results are broadly applicable. It's exactly the same in the social sciences. If you're looking at something real, it will show up consistently in repeated experiments with proper controls despite variation in irrelevant initial conditions. The phenomena that social sciences try to understand may have many more variables, but there's nothing inherently special about humanity that makes us impossible to describe.

I'm not saying humanity is impossible to describe (on the contrary!). your physics example is too different to what happens in social sciences, it's not a good analogy. you can largely control the conditions under which you throw the balls, while if you're, for instance, studying the recent socio political events in the USA, things are given and then you need to rely on similar past event to understand current ones. even in fields like psychology it's too difficult to even measure initial conditions and outputs. that's why you rely on qualitative analysis.

Reply on news.ycombinator.com