Hacker News Reader

None of those factors go into the scoring fwiw. They are just informational.

The scoring is done to a rubric, like a teacher would grade an essay, on various criteria for good & bad writing.

None of those factors go into the scoring? What's the point of showing them then? What information are these factors providing if they aren't being used in the rubric? Without knowing the rubric (which isn't provided), these scores are baseless to me, and I don't know how to use them. Blackbox numbers are not useful.

All the judge outputs (including rubric) and model outputs are in the samples reports.

Sorry you don't like the displayed metrics. I find them very useful / revealing of the things I'm trying to measure with this benchmark.

These are important things to look at because they're where LLMs typically have trouble. People complain that their stories are too short, that they use stock phrases and purple prose, that they start to repeat themselves at a certain point, and that quality tends to fall off. Doing well in these problem areas doesn't guarantee good writing, but it does allow for it.

Reply on news.ycombinator.com