As to your very last point, it isn’t my “job”. It’s yet another task that I take on for no recognition, nor additional pay - as is much of academic life.
So I'd ask you do one of two things, either:
- Review a work with the diligence and care that you wish someone would give to you
or
- Don't review
I'd also appreciate it if you openly recognized how stochastic the system is and that if/when you become in a position where you need to evaluate someone, that you remember this and take it into consideration. It has a lot of value to you too, since if the metric is extremely noisy it doesn't provide you value to heavily rely upon that metric. Look for others.
But, people pay me to do a job. It's not in my contract in any clear terms, but to do my job well I need to do all those things; and I like doing my job well.