Yeah in my work I’m observing a clear hierarchy:

* They are worst at critiquing what has happened in the same model and same conversation

* they are slightly better at critiquing the output of another model it not given all the conversation

* they are better at critiquing a weaker or earloer model by the same vendor

* they are best at critiquing the output of another vendors model

I imagine it is about correlated bias in the training data and regimes etc.

I often run adversarial models in dyadic reasoning. But then have a feedback loop to get that same outcome from a cheap model. A kind of poor man’s distillation :)