Hacker News Reader

Comment by rocqua | Hacker News Reader

That 85% 'accuracy' is ambiguous.

I couldn't get into the article, but if that number includes false positives, that's not really practical. Suicide is really rare, so it could just be that it picks a 15% of the population that includes al suicidal people. That means the vast majority of those positives are false positives.

Basically, its the precision and recall that matters.

View on HN

I'm not even sure why people report ONLY accuracy as an evaluation metric. It means nothing - appropriately weighted F scores, precision, recall give more insight into the system.

Completely agree. Although if your binary classes are perfectly balanced then accuracy is a valid metric, but that's rarely the case.

I guess the problem is that an ROC curve wouldn't create flashy headlines.

> I guess the problem is that an ROC curve wouldn't create flashy headlines.

You can, you just need to put in a little bit of work. We don't need to specify the exact accuracy in the title.

I think machine learning people use a combination (specifically the harmonic mean) of both when evaluating:

https://en.wikipedia.org/wiki/F1_score

They should use it, but articles often drop that (or similar) numbers. It's easy to get headlines if you report 85% accuracy as it sounds good and is easy to understand. Reporting the proper numbers is harder to understand for people who are not working in the field and you'd quickly discover that the model doesn't actually work.

I wouldn't call suicide really rare, it's the 10th most common cause of death in the US

http://www.cdc.gov/nchs/fastats/leading-causes-of-death.htm

Although nowadays you have to view these statistics carefully, for X, Y, and Z causes of death inevitably rise as I, J, and K get knocked down by modern medicine, all we've done to reduce many types of accidental death, from 5 gallon buckets to cars, etc.

But, yeah, in this case I agree it's a big problem in the US.

Reply on news.ycombinator.com