OpenAI Realizes It Made a Terrible Mistake

msn.com

5 points·galaxyLogic··5 comments

I was once working with an E-Learning company and proposed that our multiple-choice tests should give -1 for the wrong choice, 1 for correct choice and 0 for no answer.

Instead they wanted to only give poinhts for correct answers, not penalize wrong answers. That obviously leads to and promotes guessing. Why did they want it that way? I think they wanted to show that with our product people actually learned the stuff and thus was worth paying for. You could pass the test by making "good guesses".

Something similar seems to be going on here. AI companies want their LMS to get good scores even when they don't know the answer, in which case they guess. That is bad because they don't tell us when they're guessing.

I think it should be OK for LMS to guess but only if it clearly tells the user it's answer is just a guess, when it is.

> I think it should be OK for LMS to guess but only if it clearly tells the user it's answer is just a guess, when it is.

Or, alternatively, it shows us the confidence level of the answer, e.g. a value between 0 and 1, with 1 being 100% confident.

That would work for me.

That's a good idea

It depends on what you want. We gave our students voluntary additional exorcice, and we gave them 1 point for a correct answer and .5 in the second guess. The idea was to encourage them to learn from their first mistake (and perhaps some clue, like "remember the parenthesis" or "(a/b)/(c/d) is not (ac)/(bd)")

That's a great idea

Reply on news.ycombinator.com