Hacker News Reader

Comment by justin_vanw | Hacker News Reader

The point is that any theory made after events are observed cannot use the events it is explaining as evidence that it is correct.

The reason is that given any dataset, you can come up with thousands of stories that explain the data perfectly, but have no predictive power. This is also called 'overfitting' in machine learning contexts.

However, a prediction made before events can use those events as evidence of the correctness of the theory.

What is the evidence for the theory in this article? Well, it is that it explains past events. But those events can't be used as evidence for the theory, and if that is all there is, this is a 'just so story'.

Another way to say this is: "The fact that a theory does not conflict with existing observations is not evidence that it is correct."

View on HN

> The point is that any theory made after events are observed cannot use the events it is explaining as evidence that it is correct.

That's complete nonsense. Theories are almost always made after collecting observations.

They may be confirmed using future observations, or by statistical analysis, however.

He's right. It's the same reason why the FDA forces you to state your hypothesis prior to running a clinical trial. If you just run a trial then it's pretty easy to cherry-pick the data that supports your claim.

There is nothing wrong with looking back to see if a variable explains an observation. However, the true test of the hypothesis is predicting future events.

They don't have to be future. They just have to be unknown when the hypothesis is formulated or selected.

Because we can't trust people, future is a safe and sufficient choice. It's not necessary. (That's why machine learning people use hold-out sets.)

But if the Fed and economists do that, people embrace the conclusions as scientific :p

Well then you would need to find an incidence of someone stating the theory before it actually happened.

Before the theory happened? I don't follow.

Edit: If you mean that you need to state the theory before you make the observation for the observed data to be valid evidence for or against the theory, you are correct, this is very nearly contrapositive of the scientific method.

This is economics, not physics. Since we have institutions like the Fed and "fiscal policy" (where and if it differs from "government spends more money NOW") you cannot have a widely accepted economic theory without influencing the economy.

Two things immediately come up when looking at economic theories:

1) a LOT of them (the ones "followed") are "by" high-level Fed officials. Biggest example: Keynesianism.

2) They all seem to have the same conclusions (government needs to spend more, either directly or through the fed or finance dept)

3) because these people want to but can't be bothered to manage an economy (meaning hundreds of thousands of small companies) they massively advantage large corps over everyone else.

Reply on news.ycombinator.com