Hacker News Reader

Comment by DasCorCor | Hacker News Reader

I could have acheived better accuracy (87%) by simply calling everyone a non-murderer.

That's why naive "error rate" is not useful and not used measurement for any classification that's not like 50/50 but more like a "needle in haystack".

A more useful measure is F-score (https://en.wikipedia.org/wiki/F1_score) according to which calling everyone a non-murderer would give 0%, but the method described in article ("81.29% overall accuracy, 80.00% specificity, and 81.48% sensitivity") would give 81%.

two people have the opportunity to invest in 20 startups. the first person says no to all of them, and the second person invests in all 20 of them. all but one go bankrupt. the first person proudly proclaims "i can avoid bad investments with 95% probability!"... and the second is a millionaire.

Nobody became millionaire by calling everyone a murderer.

just pointing out that the "success" of your model depends heavily on how you choose your evaluation criteria. to me, 87% correctly classified seems irrelevant because you can just change the distribution of murderers and non murders and your model would fail. it seems more interesting to me to focus on, say, the rate the model successfully classifies murders.

The risk structure is different. When you make an investment in a company that turns out not to be successful, you wasted your money. If you persecute someone who would never have murdered anyone, you may have done great and irreversible harm an innocent man.

The rate at which it correctly classifies non-murderers is more important for this measure to be useful.

Reply on news.ycombinator.com