If you do it right…

That’s the problem: these metrics often come from overfitted or in-sample data, and are completely unrealistic when it comes to expected generalization performance.

I’m at the point where I never trust performance metrics anymore. Or rather, the worse they are, the more I trust them!