When writing feedback, you simply look back at 20 other interviews where you asked the same question, and write "candidate X's performance on this question was was in {top,bottom} {5,25,50}% of ... at this level".
You also have no idea how other interviewers will score, and if your score is 1.5 when everyone else's is a 3.9, it's likely that the HC will ignore you.
The HCs do see multiple feedback from you and your peers, so they build a model of your feedback for themselves: P is a soft interviewer, so we'll adjust his score down but Q is a very hard interviewer so we'll adjust her score up ;-)