I think this is a concise and good overview of ML model evaluation. I was recently thinking along the same lines and posted it on my blog here: https://scienceofdata.org/2020/02/23/ml-classifier-evaluatio...

You have nice graphs and cover regression models too. I use mostly these when evaluating models I've built, I also use the Kappa statistic which can be a useful summary.