Comment by DINKDINK | Hacker News Reader

Comment by DINKDINK | Hacker News Reader

Relevant stats:

>The algorithms predicted a clinical diagnosis of ASD with high specificity, sensitivity and positive predictive value, exceeding 95 percent at some ages.

More about the metrics you care about[1]

Edit: Many people in this thread are talking about bayesian stats that it appears they don't full appreciate or understand. They're saying that 95% statistical accuracy is commendable. 95% sensitivity and 95% specificity aren't good enough to use in broad tests. Why? Autism has a 1/68 likely hood[2]. Meaning if you had a sample of 100 general-population people, tested them with this test, the likely hood of someone who tests positive for the test is actually positive (positive predictive value) is a measly ~20% (that is Probability that you have the condition given you test positive). Play around with these more at the following app: https://kennis-research.shinyapps.io/Bayes-App/

[1]https://en.wikipedia.org/wiki/Sensitivity_and_specificity [2]https://www.autism-society.org/what-is/facts-and-statistics/

Thanks for writing this summary. Plenty of people get the wrong idea when it comes to test accuracy for topics like health diagnosis.

In the case of this particular topic it does seem like the outlined test could be another tool that doctors could utilize. If for instance a child has shown a change in developmental milestones then that observation comes with it's own (somewhat doctor specific) sensitivity and specificity. That information could be combined with the EEG test to improve the overall doctor+test accuracy. Nothing's going to be perfect, but the outlook is a bit more positive than presented in your example.

>Nothing's going to be perfect, but the outlook is a bit more positive than presented in your example.

Ideally what would happen is that the doctor would use their judgement to narrow down the candidates who the test is applied to who have strong priors of Autism. That would substantively increase PPV. You'd need a ~50% prevalence before you get to 95% PPV

Thank you for this comment; it has been an eye-opener for me. I always eyeballed those statistic and assumed 0.95 is good; now I've realized that it's not specificity, but (100%-specificity) that matters.

Reply on news.ycombinator.com