Cambridge professor on how to stop being manipulated by misleading statistics | Hacker News Reader

Cambridge professor on how to stop being manipulated by misleading statistics

qz.com

80 pointsOsiris3015 comments

"We know, for example, that “relative risks” can be used to look impressive."

This is, sadly, extremely true and even more sadly, very often used. 5 times more likely to die from x, sounds impressive, risk increase from 0.001 to 0.005 percent does not. Newspapers want to ... sell newspapers, and sounding dramatic is part of it. If every person capable of reading "Bad Science" by Ben Goldacre did so, this would be a better world. It's by the way the best I've seen to give the amateur an understanding of statistics, studies, being critical, manipulation by media, et cetera... (while being entertaining and an easy read, a frequent present of mine to other people).

"Depends on what you mean by rational. I don’t like that word. You could use other words like “value-congruent,” which fit in with what people feel is the appropriate value. Those are the decisions they will make and not regret in the future."

"Value-congruent", nice one. "Rational" is indeed one of those empty words that is thrown around.

One thing I also find incredibly important: People have to understand science is a METHOD, not an AUTHORITY. Because people don't listen to authority, but in many cases they can't argue much with reason. This is incredibly important, a lot of the anti-science sentiment stems from viewing science as an authority, and this is extremely stupid.

It should be noted that there are reasons, both good and bad, for why we use relative risks: http://journals.lww.com/epidem/Pages/articleviewer.aspx?year...

Also, at this point, relative risk models are much more well developed. While we'd all love absolute risk measures, if you have to choose between an absolute measure with known confounding we can't handle with current methods, or a relative measure that can, the choice is much less clear.

At the end of the day, as noted in the article, different data has to be presented in different ways, or actually the same set of data may be presented in different ways, it depends on what you want to focus on, what exactly is your subject of interest, et cetera. But, and that's my point, if the entity responsible is not interested in how to represent data or a certain conclusion one has made as clearly as possible to the public, but to sell newspapers and get clicks[1], then what do you expect?

So it is not my intention to disregard any form of presenting/visualizing data without a specific example or proper context.

[1] It is my personal belief that Social Media, Twitter, and the short attention span of the modern age all aided extremely in this. If you look at "sophisticated media outlets" at Facebook, they are playing the click-bait game nevertheless. How to be on Facebook or Twitter and not be a tabloid?

This was actually more of a general note than one addressing you specifically. I've seen "Relative measures are wrong and bad and you should feel bad!" enter what I'll call the "Educated Layperson" lexicon regarding statistics, while ignoring that there are often very good reasons for why these choices are made.

For more reading on the topic, I'd recommend "How to Lie with Statistics"[0]. It's a short read (144 pages), with mainly tongue-in-cheek instructions on how to mislead.

Most of the usual tricks ('drop the axes', percentage-points, etc) are there, but there are many other, less obvious tricks.

One of the cooler arguments in the book is that it's easy to lean on someone's implicit assumption of volume to modify their understanding.

If you inflate a 15% increase in house spending to look larger than it is, drawing pictures of houses that are 15% wider will make people intuit a 50%[1] increase, despite reading 15%. The author suggests that even if you're incredibly clear with the text surrounding the charts, people still use the charts to understand the scales of change.

[0] http://amzn.to/1RmCVmL (disclaimer: affiliate link) [1] 1.15^3 ~= 1.5

>"I got very grumpy at an official graph of British teenage pregnancy rates that apparently showed they had declined to nearly zero. Until I realized that the bottom part of the axis had been cut off, which made it impossible to visualize the (very impressive) 50% reduction since 2000."

Don't understand. I'd think not truncating the y-axis would make it more difficult to see a change.

>"I thought people would know that 3 out of 100 is equal to 3% is equal to 0.03. But they are very different!"

Don't understand. Those numbers are all the same by definition.

> >"I thought people would know that 3 out of 100 is equal to 3% is equal to 0.03. But they are very different!"

> Don't understand. Those numbers are all the same by definition.

They are ways of representing that same number but people respond to them differently.

So the numbers are not very different, just erroneously perceived to be different (according to psychologists). I guess it isn't as odd as I thought after skimming, but I still don't like it.

Numbers are not different mathematically. Written representations of numbers are different psychologically.

If you start a graph at like 50 on the y axis it's easy to make data look more pronounced. The official graph showed they dropped to "zero", presumably a bigger difference than 50%. Like it would look like the line dropped to zero, but really the graph started higher than zero on the y axis to begin with.

Idk about the second one.

Thanks. I didn't think that he meant they were exaggerating the change, but you seem to be correct.

See this: https://twitter.com/d_spiegel/status/707860293683126272

A good guideline for whether to truncate a y-axis or not is that the conclusion that a reasonable and knowledgeable researcher would draw after looking at the data should be clearly implied based on a quick cursory glance at the chart. If the chart implies an unreasonable conclusion based on a cursory glance, then it might be said to be misleading. A chart's purpose is to convey information quickly, not to convey information precisely -- for that, a table is often better.

In the example, the true effect is that there was a ~50% decrease, which is newsworthy and interesting in and of itself. Yet, intentionally or not, the chart implies a ~100% decrease based on a quick glance, and so fails to highlight the conclusion that a reasonable person would draw from the underlying data. In this case, I would say a y-axis from 0-60% would be appropriate.

There's a fun video he created with Cambridge University a few years ago, called "Professor Risk". Worth a watch (6 minutes) https://www.youtube.com/watch?v=a1PtQ67urG4

Takeaway from the video "I've come to the conclusion that one of the biggest risks is being too cautious."

Another source I can link to when people in comment sections blindly state, "y axis should start at zero!", which is almost as annoying "correlation isn't causation". Context.

Reply on news.ycombinator.com