The Discovery of Statistical Regression

It's used everywhere, and possibly overused, too. I worked at a place where they still haven't discovered regularization. Everything is literally the same 200 year old tool.

The problem with that is of course that when you have a hammer everything looks like a nail. Works fine if what you're looking at satisfies the assumptions, but once it doesn't who knows what you'll get.

I find the statement "Those predictions are optimized so that, for any point in the data set, the model’s error multiplied by itself (squared) is minimized" misleading. What's being minimized is the sum of the squares of the errors.