The declining authority of statistics and the experts who analyse them

We should hopefully be in agreement that using data to make decisions is preferred to winging it. We should also hopefully be in agreement that we should trust people who devote their life to studying an issue over someone new who merely has gut feelings about it -- and hence is even more biased in many additional ways.

The thrust of your argument is valid -- summarizing data inherently leaves things out. However, the solution to this is not to give up and throw statistics out the window. And the solution is definitely not to unfairly discredit people who devote their lives to these issues.

Rather, if you see statistics, wonder how they were generated and why some data was included and others thrown out. By being open and candid about the decisions made, the statistics are meaningful. Just giving numbers without providing details of how they were achieved is not valid statistical work.

From my experience, most scientists (and myself in my work) care about the truth over everything else. I use statistics to better understand my data. My objective is always to choose the most sensible path in analyzing it that allows me to most accurately answer the questions I pose.

> there is an increasing body of evidence that experts

Is this "body of evidence" non-statistical?

However, everyone who's trusted any technology that accounts for quantum effects (e.g. modern computer processors) has trusted statistics. Everyone who's ever received a modern medical treatment (of any kind) has trusted statistics.

The hard part is knowing when you should, and when you should not, trust statistics. Frankly I don't think there's anyone who can do that 100% of the time -- but I think most people could probably learn to do well enough to benefit. The learning part is tough, though.

People (rightly, IMHO) realize that political activists sometimes work under the guise of scientific researchers. It shouldn't come as a surprise that this kind of conduct erodes confidence in the field as a whole.

Sadly, the baby has a nasty habit of being thrown out with the bath water...

This is true (and conversely, a prediction can still be "wrong" even if the outcome matches the prediction, if the prediction expresses the wrong degree of confidence).

However:

> when statistics can only tell us something about it when repeating it

This is only true if you take a pure frequentist approach to probability. The Bayesian interpretation of probability has no problem with defining probabilities over single-occurrence events. (And even frequentists have a way of dealing with this, although Bayesians would argue that in doing so they're essentially adopting a form of Bayesian probability).

This leads to a situation where central bankers are publicly saying "hmmm, we've done massive QE but there's no inflation, what a mystery" whilst the average punter is seeing housing and stock market bubbles inflate right in front of his eyes. It causes people to tune out: headlines like "inflation statistics hit new low" are placed right next to "house prices reach record highs".

But, that mistrust has been around for a very long time.

A big part of the mistrust is not in the numbers themselves, but in the people who trot out the statistics and the way they carefully cherry pick numbers that support their cause, ignoring other numbers.

Even without those issues, you still have the issue of people you survey not telling the whole truth... or surveys being poorly designed so that the question people think they are answering is not what was really being asked.

Data is hard. Just because you have some survey data and (maybe) did something statistical doesn't at all mean you should listen to it.

It was a favourite trick of British governments in the 80s and 90s to continually change the definition of unemployment, both to reduce the headline numbers and to prevent the construction of comparable data series.

My mom, a high school chemistry teacher, used to say, sarcastically: "If you want your data to fit a straight line, collect two points." And if you only collect two points, you've made a tacit assumption about your model.

In general, the form of your data set (what things you vary from one measurement to the next) determines the form of the model that you can fit to the data. This is particularly evident if the model function is a low order Taylor polynomial.

The interpretation of the results is another chance for hidden assumptions. For instance, referring to the inflation rate as "inflation" when it doesn't include the cost of higher education and health care, for instance, represents a bias that's easy to overlook.

Most people have a pretty poor idea of just what and how much they are actually eating.

How do you take a random sample of Americans? If you call landlines, you'll be heavily biased toward the demographic that has landlines. If you call cell phones too, you'll be biased toward the demographic that has phones. You'll also be biased toward people who answer calls from numbers they don't recognize, and are willing to spend ten minutes talking to a stranger about whatever.

You can survey people in the street, but then you're biased toward people who go out more, and again biased toward people willing to spend ten minutes talking to a stranger.

Obviously they know about these problems and try to compensate for them. And I'm sure they usually do a pretty good job of compensating. But that compensation involves a lot of modeling and assumptions.

Also, "Data Science" doesn't refer to Science with Data, but the study of data analysis itself, and for that purpose is a reasonable albeit often unnecessary term.