> We don't really need truely objective assessments. As an analogy, imagine we randomly added or subtracted 3 minutes to the finishing times of all the runners in the Boston marathon. We wouldn't have a truely objective assessment of each runner, but it's still a useful guide about who is likely to beat whom in the next marathon.

I don't think the analogy fits. There's no single metric (like runner speed/time) to evaluate competence.

> This problem can be alleviated to a large degree by risk-adjusting the outcomes: For each patient, estimate the most likely outcome and compare the actual outcome to the estimate.

Agree.

> I don't see this as an insurmountable obstacle. For example, you could have an independent body randomly sample the reported outcomes to check that they are accurate, and apply some kind of penalty if they are not.

That would be a good solution. Part of the problem is finding a truely independent body. Medicine is over-run with various governance and regulatory bodies (several of which have been shown to be little more than rent collectors). And again, there's the problem of deference to eminence combined with small in-bred communities within each speciality or sub-specialty (who would likely be the only people with the training to evaluate their peers reliably).