I don't think you can calculate E^2 without the "true model", which you practically never have. The code uses the "true model" too: https://github.com/johnmyleswhite/r_squared/blob/master/util...

I guess the post is similar to Anscombe's quartet [1]: a warning not to blindly trust summary statistics.

[1] https://en.wikipedia.org/wiki/Anscombe%27s_quartet

My main issue with R^2 is that it is an artificial indicator. It has no business meaning. The best alternative is to build a metric that is related to your business issue. How many $ do you loss if you're wrong by 1%? Or by 10 units? Do your business loss is linear in your error? Is it symmetrical?

These are the kind of questions you have to ask yourself. Defining a metric is hard, and there is no good shortcut.

The author says that E2 is "generally unmeasurable". Indeed, the true model will be unknown in most cases.

In my opinion, it is not about using E2 or other alternative measure to R2, but about being aware of the significance and validity of R2 and not blindly assuming that a higher value always means a better fit.