Some of the problem could be mitigated by having multiple reviewers score answers, and taking the average of the score.
Of course this would increase the cost of grading such tests...
Of course this would increase the cost of grading such tests...
Questionnaires were printed and sent out in bundles to cooperating village chiefs, who distributed them to the peasants. Literacy was at a sufficient level that most were able to complete the single question, which was, of course: "How long do you think the Emperor's nose is?"
When the forms were collected, mathematicians added up all the values, and divided by the number of forms. Thus it was known that the length of the Emperor's nose was 6.734602 cm. The complete set of data was of course preserved, and many years later, with advances in statistical understanding more advanced mathematicians pointed out that fringe values - obviously the product of deranged minds - were distorting the honest opinions of the rest, and by eliminating them and using the very latest numerical modelling techniques, the mathematicians corrected this value to 4.980403 cm. To this day, no-one has produced a better estimate.
the china emperor: that's maybe because there's no photograph of the emperor circulating. if there is, one can make comparison to eyes, age, coin, ground-legal-measure etc and clean out the outlier data. averaging the rest can give good enough estimate but not super accurate.