Statistics have lost their power (2017)
theguardian.com
theguardian.com
It's easy to decry:
> Presented with official estimates of how many immigrants are in the country illegally, a common response is to scoff. Far from increasing support for immigration, British Future found, pointing to its positive effect on GDP can actually make people more hostile to it.
... but the underlying belief appears to be closer to:
> If you live in one of the towns in the Welsh valleys that was once dependent on steel manufacturing or mining for jobs, politicians talking of how “the economy” is “doing well” are likely to breed additional resentment. From that standpoint, the term “GDP” fails to capture anything meaningful or credible.
The first statement makes no sense in and of itself, and it would seemingly suggest a knee-jerk reaction against the very idea of data. However, in the context provided by the second statement we see a different view: a core distrust that "average well-being" (represented by the statistic of choice) in fact represents the common good, suitably defined[1].
As a result, I feel that the article's later musings -- that we're merely collecting the wrong kind of statistics -- misses the core point. If we start reporting on underemployment, then the right circumstances might come along to discredit that indicator, and so on. The article begins to grapple with this issue in its digression into big data (and how there are no national agencies for it in the manner of the ONS), but even there the author deals with a practical problem of said data being more useful in the short term than long term than the core problem:
Ultimately, if society as a whole does not have a shared view of the 'common good', then _no quantity at all_ can measure it. The question of "is this good?" becomes fundamentally unanswerable in a democracy with an identity crisis.
[1] -- That could mean "median well-being," "well-being of people I know / in my region / like me," or "personal, individual well-being of the speaker."
If I am not doing well and some journalist or some economist tells me the GDP has grown a lot and everything is great, I am going to feel resentment toward that person. Doubly so if they see my less-than-enthusiasm and start talking about the “common good.” Why? Because the implication is either that there’s something wrong with me (for not keeping up with the Joneses) or that I am an acceptable sacrifice for the benefit of the common good.
Okay, it’s fine if you (the journalist or economist) think I’m an acceptable sacrifice, but don’t expect me to be happy about it, let alone agree with you. Don’t expect me to support the status quo, come Election Day. DO expect me to try to throw a spanner into the gears. DO expect me to support a strongman who says he’ll champion my cause over the cause of those who think I’m the acceptable sacrifice.
In many cases, reporters want to be dramatic, but not liars. However, the lack of deeper statistical knowledge often lead to them becoming liars, even though they only wanted to be dramatic.
"People assume that the numbers are manipulated and dislike the elitism of resorting to quantitative evidence."
If people don't believe that something is useful, it doesn't matter what that "something" is, it will start to have a diminishing affect our society. It goes for statistics and liberal democracies just as much as ancient gods and ancient languages of yore.
I doubt statistics or mathematics gained its influence because various cultures were fond of symbolic play. We study these things because we expect concrete payoff, not because of cultural affection.
So is social science lame? Or is Google insight scary? At some point a deliberate posture of mis-devaluation will come to terms with reality.
Sure I'll back that up. Thought experiment: if everyone, including all the drone pilots, suddenly magically stand up all at once, walk away, and never talk about it again, then yeah it won't impact humanity.
Is this particular scenario realistic? No. Has mathematic or chemical knowledge been collectively lost before in human history, and been eliminated from society? Yes.
> but some technologies are sufficiently powerful in their own right that it needs only a few people to recognize their power. Then the stress of competition can do the rest.
Sure, those few power can recognize its power, but if it's anything short of immortality, never underestimate the power of every other human being being a Luddite wielding a pitchfork ready to kill over progress, because it violates a cultural norm. Doesn't matter if your statistics can find a cure, societies and collective humans' irrationalities are expert suppressors of technology as both a mean and an end. Look no further than the US stance on nuclear waste reprocessing. The science and power are there. The will of people, is not. Or, look at South Africa and Ukraine giving up nuclear weapons. Power? Yes. Popular will to hold onto this power? No.
> We study these things because we expect concrete payoff, not because of cultural affection.
Exactly, because society values concrete payoff, things like "derivations from first principles" and "following the scientific method". But if society as a whole pressures people to never pursue ideologies along these lines, or turn actively hostile against it (book burning, HDD erasing).
I take this article as an alarming warning. Scientific progress is itself not an end. And it is not an inevitability. I like scientific progress and statistical tools, this article just goes to show how careful -- ethically responsible -- those of us wielding them must be when living in our society. And call out bad actors.
This is why engineering professions have ethics boards. If engineers build bridges and bad actors build bad bridges, people won't trust bridges anymore. And if people don't trust bridges, they don't get used, so more don't get built.
The greatest failure of the "modern" education has precisely been the removal of religious ethics without an equivalent amount of time working to help students develop their sense of personal ethics.
The scientific system needs to be opened up or replaced. Distributing information and experimental results no longer requires expensive printing.
People are right not to trust it, scientific journals are now the gatekeepers rather than the gateways they once were.
Wine is good for you, chocolate is bad for you, cholesterol is good for you, wine is bad for you, red meat good for you, Fluoride bad for you! I could find you a statistically significant study supporting any conclusion I want right this minute. Since that is the case, why should the average Joe believe any of them? This is made even worse by news articles and popsci blogs not even linking to the source papers, so avg Jane can't even investigate further if she wants to.
Solution is to support proper scientific procedures, support civic organizations carrying out the important tasks of vetting claims, and being an example of someone who values the correct protocols when evaluating data and making conclusions (or not making them).
It's a very slow process, and we're lucky we're in a timeline where the work of various people throughout history has gotten us here. We'll have to work to keep it going further, and literacy in math, statistics, and the scientific process is crucial.
Compare the two statements: "10 out of 80 people that took the drug lived a month longer but were extremely miserable."
Versus: "A new cancer drug doubles the average survival rates scientists say".
Which one do you understand better? Here I just made up some examples, but as a scientist I'll tell you just about all the papers I read are in the second category, it takes an unreasonable amount of effort to get the first type of infomration out of the publications.
When I use the word statistics in this conversation, I mean the use of math to analyze and compare data. This can be done inappropriately, of course, but the problem doesn't lie with the principles of math (including statistics), but the agent using them inappropriately.
And medical (or biology related) statistics are used most inappropriately, due to various cognitive biases and financial incentives. But the problem is not statistics, and it's still the best tool we have unless we can find a Magic School Bus to shrink us down and go observe the chemical reactions in real time. But I know that stats are abused, so anytime I see a fantastical claim, especially in medicine, I approach it much skepticism.
Psychology journal bans P values
https://www.nature.com/news/psychology-journal-bans-p-values...
I happen to agree with the sentiment expressed above that the entire concept of hypothesis testing is fundamentally flawed. This is what you call the "best tool". It is not, it is the tool that is used because it allows for the most fudging of results.
There are endless ways to be unintentionally obscure, to semi-intentionally obfuscate or to just flat out lie. And it very hard to tell these apart - one would need to dedicate more effort than usually worth.
And obviously (sweeping) results from health related experiments should obviously be met with skepticism considering the complex nature of the problem.
But that's practically all they're used for!
IMHO it actually suggests we need to improve the average person's educational level
walk into any university, go from one faculty office to another, ask each professor what a p-value is and the overwhelming majority won't give you even a half-decent definition (I did this and can attest to it)
you know why? because p-values are far more complex than people care to admit. Most statisticians cannot properly use it! There are ongoing "ideological wars" over what it actually means. Even the founders Fisher etc argued all the time about them. If you used p-value correctly, you would almost never be able to find anything, not even valid results.
So people have to "bend" the truth a little bit. Nowadays more often than before, the truth gets bent out of shape.
But yes, society doesn't function well when people are corrupted.
Your distribution is not normal - it almost never is, now you have to argue that the method is still robust - there is no proof that is indeed robust as you claim. Then your measurement is a bit biased, see on Tuesdays the measurements do turn out a little bit differently, but most often does not matter, so you skimp over that as well.
Statistics is the wrong approach because if you did everything correctly and stuck to the letter of it you only end up hurting yourself. You'll make fewer of the valid discoveries that you make when you bend it a bit.
This is the fundamentally wrong thing about it! Strictly sticking to the rules will make you weaker. Bending statistics a little leads to more valid results.
If it is not normality then something else. Every statistical measure makes assumptions, most of which never quite hold true. That was my point. When someone uses statistics they always fudge things a bit. Many times does not matter, if you were stickler for "precision" it would not work.
It is it's own thing, the art and science of making data and observations into information. Yes newspapers and the regular pundits abuse it all the time, but don't pin that on statistics, pin that on those that abuse it with, or without (usually) knowing better.
Statistics are unavoidable, unavoidable enough that much of our perception seem to make use of the same statistical calculations sciententists use, only without us having the ability to scrutinize its workings.
As the old saying goes: "Better to remain silent and be thought a fool than to speak and to remove all doubt."
https://slatestarcodex.com/2014/04/28/the-control-group-is-o...
Basically it boils down to the fact that studies & statistics used are still very, very, prone to human bias.
Data is great for illuminating things, but for arguments, it's usually cherry picked. I do think that writers for the guardian vastly underestimate the intelligence of people who don't agree with them. I generally mistrust statistical arguments because someone forearmed with statistical talking points has usually already signalled their intransigence. Data is nice, but alignment of interests and incentives is better.
It reduces to, "according to criteria and data I have selected, your argument does not bear out." It is a passive appeal to the authority of whoever provided the number, and not meaningful or logical information in itself.
Logical arguments tend not to persuade anyway, but to say someones argument is illogical because it does not resist a coded illogical premise (appeal to authority fallacy), is certainly a way to make them give up on arguing with you.
"Paul Krugman and other mainstream trade experts are now admitting that they were wrong about globalization: It hurt American workers far more than they thought it would."
But nobody cares about that, because nobody cares about statistics anymore. They want that dopamine hit of outrage, so they read the FP article, that deliberately misrepresents the argument (notice how "manufacturing" suddenly becomes "workers", despite the fact that most workers are not in manufacturing), and doesn't give any evidence. It just quotes a bunch of people settling scores with Krugman. Even there, it mischaracterizes, for example, Krugman's review of Greider's book to heighten the outrage.
Statistics is boring, and rage is interesting, so the Internet era optimizes for rage.
> The thinktank British Future has studied how best to win arguments in favour of immigration and multiculturalism.
BF didn't set out to figure out whether immigration and multiculturalism is favorable. They only tried to figure out how to use statistics (or anything else) in support of their foregone conclusion.
Maybe the headline should be "Statistics have lost their usefulness for propaganda."
"doing the same thing over and over and expecting a different result."
probably the most common mistake students everywhere make. They keep taking more courses - it never helps just makes you more confused.
Really, that's the perfect example of why statistics are distrusted - only those statistics that align with someone's agenda get presented. And sometimes, they're not even gathered in the first place: https://www.thelocal.se/20180508/why-sweden-doesnt-keep-stat...
The article is not suggesting that statistics should not be scrutinized.
But, give them statistics that go against their pre-ordained narrative, and suddenly it is the reader who doesn't immediately refute the statistics who is the uneducated rube.
Reduces trust in science, statistics, and the media all at once.
Plus it gets misused all the time - even by statisticians that should know better.
When the statistic is a proxy for a quantity of interest, and when policy exploits ways of manipulating that proxy without commensurate change in the underlying quantity of interest. My driving policy of targeting my speedometer does a fine job of controlling my vehicle's speed even though the former is just a potentially-inaccurate measurement.
In the case of the CPI (actually the PCE for the US Fed [1]), the quantity of interest is the "[nominal] cost of living." To the extent the CPI doesn't in fact measure the cost of living, the easy solution is to change to a price index that does measure the cost of living. Since living tends to involve buying and consuming things, a price index, suitably defined, can be a fine measure of this.
To the extent you are concerned about the exact ways in which the CPI is computed, independent measurements [2] generally reproduce the broad trends in US data, which in turn suggests there is no significant manipulation by the BLS.
[1] -- https://en.wikipedia.org/wiki/Personal_consumption_expenditu...
The other 90% challenge the sources of this article.
(With apologies you Yogi Berra)