Hacker News Reader

Comment by madeofpalk | Hacker News Reader

> you have to shuffle the order in which you grade these submissions each week, for fairness

I don't think this is fair. It's just a more randomly distributed unfairness, rather than by a deterministic factor (like the student's name)

'Fair' would be each student is assessed independently for the work they did, rather than their mark being impacted by how early or late they were marked.

View on HN

I think an important difference is that when you shuffle them, the unfairness stops being correlated across multiple assignments, so the "aggregate" unfairness over the course of the semester is much lower.

It would be essentially impossible to have something "truly" fair for open-ended questions since humans are stateful.

Maybe this is a case that AI could actually do quite well.

Manually grade the answers and identify the classes of mistakes. Then hand the classes of mistakes to the AI and ask for it to determine which answers have which types of mistakes.

Once you've done that, you just need to associate a deduction for each type of mistake and do some simple math.

what do you mean AI? you must be joking.

Imagine a question: compare bubble sort and quick sort algorithm.

Some students might mix up the algorithms, some might give the incorrect computation complexity, some might describe them incorrectly in some way.

Manually grade some (or all of) the answers by noting the kinds of things students got wrong (e.g. the above criteria). Then, feed in to ChatGPT (or your favorite alternative) the answer + the categories of mistakes to expect.

Here's a simplified example: https://chat.openai.com/share/bf801e12-51d5-4255-9968-bbf91b...

There are many notions of "fairness", many of which are logically incompatible with each other.

In this example, I think it's kind of fair to give everyone an equal chance of being advantaged. You're not hurting anyone specifically.

Is that distinction worth making here? There’s no way to “assess independently” the work of each student without some amount of randomness. But I think that’s okay, because isn’t randomly distributed unfairness just… fairness?

Reply on news.ycombinator.com