Your idea is along the lines of 'rater reliability' in the field of educational measurement/psychometrics.