Hacker News Reader

Comment by mike_n | Hacker News Reader

Do human doctors have a better rate? And at what cost/throughput?

This is the base rate fallacy.

    return “healthy”;

would have a 99.9% accuracy right now.

I think you have a good idea for an AI startup here.

For bonus points, have it return one out of every 1000 or so requests with "not hot dog."

Ahh, you need to use ROC AUC or something.

ROC AUC is somewhat meaningless when basically any amount of false positives is going to massively decrease the utility of the test.