Comment by endothrowho333 | Hacker News Reader

Comment by endothrowho333 | Hacker News Reader

Parent Full thread

It's not that you can't conclude anything from data, it's just that the things you can conclude from data are limited to only two "scientifically-sound" conclusions:

>1). There is a relationship in your data

>2). There isn't a relationship in your data

From there, you can then setup further experiments to better understand why there is or isn't a relationship -- and perhaps find the opposite is true!

It's the same reason you don't assume a theory is "true;" you don't conclude "that higher unemployment in Canada vis-a-vis the US is almost certainly driven by a lower supply of jobs, rather than as you posit a lower demand for employment" (GGP), because that's bad science!

What the GGP did is create an inference -- one based on misread data and faulty assumptions about scientific methods -- and expressed it as de facto. Now, in order for his inference to have weight, he must either support it with further evidence (DIY meta-analysis) or carry out some other type of experiment to test said inference.

Here's a very brief primer: https://socialresearchmethods.net/kb/concval.php

A very good example of this is how long it took scientists to be comfortable with positively saying that tobacco use caused lung cancer in people. (A fact tobacco companies greatly leveraged in their defense.)

Huge surveys long showed a correlation between smokers and increased incidence of cancer. But it took decades of research to rule out confounding factors. For example, it may have been the case that industrial work was causing the cancer and industrial workers just happened to be more likely to be smokers.

There is one tool that really nails causation fairly quickly which is a double blind controlled experiment. But usually in social science it's very hard to or even immoral to conduct such experiments.

For example, assigning babies at random to be smokers or not for some period would be pretty hard to carry out and certainly be immoral if you thought the smoking may lead to cancer.

You don't think this is an example where industry threw shade and we could have concluded this much earlier?

Genuinely no. Mostly I think the industry failed to corrupt the scientific process. Their attempts to do so were nakedly transparent. And they absolutely preyed upon the fact that real quality scientists were reluctant to definitively say the link was causal because they were being diligent about the fact that there was a preponderance of evidence that was correlative.

They succeeded in corrupting the political process though. I don't think you should need 100% scientific certainty to begin regulatory action. Maybe 80% or 90% of the way is good enough. The industry succeeded in requiring 150% certainty before a public health response could begin.

Sorry for being so dumb. Your answer just confused me. Is this standard for data something that is unique to social science? Or is this the same standard?

See dmwallin's sibling comment for more info.

It's standard for all data, but must be enforced more strictly in social sciences where controlling for all variables (let alone knowing what they all are!) is impossible, and your correlations are more likely to be pure chance.

See: reproducibility crisis in social sciences

What would you say is better: him using an imperfect study, or you just expressing your opinion?

Full disclosure: I am affiliated with myself, and represent the views of me, myself, and I. I am very biased.

Me expressing my opinion, but I don't like either. So I try to steer away from expressing "my opinion," and sticking to only things that have a high probability of being true, which in this case, is the GGGP using an imperfect study incorrectly.

Reply on news.ycombinator.com