Why Do Models That Predict Failure Fail? [pdf] | Hacker News Reader

Why Do Models That Predict Failure Fail? [pdf]

fdic.gov

42 pointsluu24 comments

> Using data from the Panel Study of Income Dynamics reveals that the relationship between predictors typically used to predict mortgage performance and the subsequent arrival of mortgage default trigger events has exhibited significant intertemporal heterogeneity.

A pretty roundabout way of saying that the future was being predicted by extrapolating the past, and that the actual future was unlike the past.

> Interestingly, diversifying the macroeconomic conditions from which the training data are sampled did not generally improve the out-of-time performance of the machine learning methods that we considered.

Mildly interesting indeed, but not quite surprising given that the macroeconomic conditions form a pretty large space. All sets of conditions for which data is available form only a tiny subset of all possible conditions, so increasing the training data from (say) 0.1% to 0.2% of possible inputs is not going to move the needle much.

In this instance in particular (mortgage default prediction), generating better predictions will produce shifts in the very conditions observed. The possibilities for emergent behaviour in a system as complex as the mortgage market are very large.

To rephrase... because the models predicting model failure (often) use the same assumptions/knowledge as the original models, so they both fail at the same time once assumptions become invalidated.

"intertemporal heterogeneity in the relationship between variables that are frequently used to predict [...] imply that model instability is a significant source of risk for [folks] that rely heavily on predictive statistical models and machine learning algorithms [...]"

No shit, Sherlock! aka "overfitting"

I don't mean to deride the actual research that was done, but I would hope that people understand this general property better before jumping on the bandwagon to use statistical models -- without needing the benefit of hindsight.

A true future predicting algorithm should be aware of its existence and the affect its prediction will have over the receiver / receivers.

This is the gist of the Rational Expectations wave of economic theory.

https://www.nobelprize.org/prizes/economic-sciences/1995/cer...

Consider the problem of predicting an agent's binary choice. If an agent follows a strategy of always choosing the opposite of what the algorithm predicts, then the algorithm's prediction will always be incorrect. A "true future predicting algorithm" cannot correctly predict the actions of certain kinds of agents, as long as those agents can observe the prediction.

There will always be probability involved, the algorithm’s job will be also be to “wrap the prediction” in a way receiver thinks will it will benefit from ( negotiate ) . Hence leading to following the prediction.

Hari Selden had a pretty developed concept what people can know about an algorithm that anticipates their behavior.

They got mad ?

"Violence is the last refuge of the incompetent." - not actually Hari Seldon

In physics, I believe this is called the observer effect.

Indeed, shouldn't that be the point?

Yes, but that would make it infinitely hard for the algorithm as the output is depending on who is receiving the output. Hence the algorithm should have perfect understanding of environment, receiver and equation between the environment and receiver.

There's an old saying that goes something like there are countless way to fail, but only a few ways to succeed. Perhaps there's more complexity in modeling failure?

All happy families are alike; each unhappy family is unhappy in its own way.

https://en.wikipedia.org/wiki/Anna_Karenina_principle

I usually think of it as being a reflection of entropy. There are innumerable configurations of the plastic and steel that makes up my car, but only a vanishingly small proportion result in drivable vehicles.

Yes, if it was that easy to predict failure, it would be easy to avoid it.

“The Lucas critique, named for Robert Lucas's work on macroeconomic policymaking, argues that it is naive to try to predict the effects of a change in economic policy entirely on the basis of relationships observed in historical data, especially highly aggregated historical data.“ - wiki

This was pretty much proven for stocks too

Taleb's preachings come to mind

How exactly?

Excerpted summary from here [0]

Difficulties of Prediction Ultimately, our world and future are unknowable and unpredictable because of various factors, including:

• The role of chance and luck in inventions and game-changing discoveries

• The inherently unpredictable nature of dynamical systems and variables that behave in inconsistent or nonlinear ways

• We can’t make accurate predictions with flawed and incomplete information.

The combination of the 2 sets of factors above means that: humans are terrible at prediction, yet we keep making predictions without realizing how frequently we’re off the mark. Taleb calls this the “scandal of prediction”.

[0] - https://readingraphics.com/book-summary-the-black-swan/

These are characteristic talebian points (it's good that you read a Cliff's Notes, I read the whole thing and what a pompous gasbag) but there isn't much that's specifically talebian to what you said.

People make predictions because they judge that it's better to have bad predictions than none at all. The probable error in prediction (e.g. the "cone of uncertainty" that becomes wider and wider as you go into the future in the simplest, best-performing econometric models) is somewhat undersold, but the general understanding is that this is still a Pareto improvement.

Bicycle helmets are also no good if you get hit by a truck.

> People make predictions because they judge that it's better to have bad predictions than none at all.

Some do. (Some make predictions because they mistakenly believe that they can predict accurately.)

But is a bad prediction better than none at all? Depends on how much you rely on it. The more you rely on it, the more it matters that it is accurate.

> But is a bad prediction better than none at all? Depends on how much you rely on it. The more you rely on it, the more it matters that it is accurate.

Sometimes (in a iterative process, for example) all you need is a starting point. But some iterative processes can get stuck in infinite loops with a bad value. So sometimes a bad prediction would be better than none at all because you just need a starting point. But sometimes a bad prediction would be worse than none at all because you'd be in a non-halting process.

I'm more in the Tetlock camp when it comes to forecasting the future. One tenant in his book Superforcasting talked about a car heading towards a brick wall.

At the current velocity, there will be a catastrophic event to the driver. However, what do people normally do in these situations? With proper prudence, they come to a safe stop before the wall. Some will, at the last moment slam the brakes and come to a sliding, screeching stop before hitting the wall. Another set will swerve and wrap around a tree or end up in a ditch. While a very small amount will actually never hit their brakes and hit the wall at said velocity.

The initial data statement regarding velocity is technically not wrong, at all. Its extremely logical and true. However it lacks taking into account the human factor. Humans are not equations. All prediction methods regarding human action revolve around math, loosely or strongly. We dont possess math that accurately depicts human actions. Folks like Taleb are so far removed from having social skills or friends, he forgets other humans exist. Perhaps it's a sociopathic or psychopathic tendency where they see no value in other humans. I always forget which one is which.

Now, we can argue economics is a study to do this in a narrow field. Which I agree to. However, the best educated economists who aren't trying to sell you a get rich quick scheme all say roughly the same thing, "people are cray cray".

Poker is also a good example of this logic. Particularly the meta play the experts are at. Poker is beyond the standard statistical average of hands, which is all there is to blackjack. You still have the human element. That's what makes it more difficult.

People like Taleb try to over simplify concepts that scare them because they dont like to be wrong. Tetlock proved in various gov studies that people can become substantially BETTER (obviously not perfect nor does he claim to have a perfect solution) than average guessing by embracing the chance of being wrong. When embracing the potential of being wrong, you list out the potentials, the whys and then figure out the catalysts. Instead of having a single cone of prediction that widens over time, you have multiple smaller ones based on potential future actions. You apply confidence rates to each one as part of your prediction process.

I... dont remember why I got on that rant. I remember I wanted to make a joke that Taleb is an idiot and essentially agree with you, but wanted to give a good reason why along with it. Moral of the story, forecasting the near future is possible, but takes a lot of work and research to do it effectively. Dynamic and complex systems are not impossible to understand if you're willing to put in the work and in doing so, you will only be "so wrong" rather than blatantly wrong... second moral Taleb is a twat.

To use the past to predict the future, you are pretty much assuming all the factors are pretty much constant and that there will be no new factors that can adversely affect it.

Reply on news.ycombinator.com