Hacker News Reader

Comment by mb7733 | Hacker News Reader

> One of the issues with using a z-score for count data is it assumes the data are normally distributed

A z-score is just the number of standard deviations away from the mean a data point lies. It doesn't require any assumption about the distribution of the data.

If you wanted to make some inference, like maybe about the likelihood of observing some z-score given a hypothesis, then you would need to assume a distribution.

But, nothing the author does assumes a distribution, normal or otherwise. It doesn't mention probability at all as far as I can see.

View on HN

When you decide that a value N standard deviations away from the mean is an anomaly that must be investigated, you implicitly make an assumption about the underlying distribution of the data.

Exactly, it’s a smuggled assumption. A standard deviation is the second moment of a normal distribution. A negative binomial is a much better model for count data: https://statwonk.com/poisson-gamma-negative-binomial.html

The simple approach in the article is no different if you imagine it as a negative binomial. The article does not assume any distribution, it's entirely nonparametric.

That's not the case. Looking for |z|>t implicitly assumes the distribution is symmetric and unimodal for a start. As in, the method would break if those assumptions were violated.

Non-parametric means more than just avoiding talk of distributions.

Good point about symmetry & unimodality. You could easily construct a bimodal example where the method fails entirely.

In my previous post I said "doesn't assume any distribution" I meant it doesn't depend on any particular distribution function.

But in the post above that I was sloppy and said "no assumptions about the distribution at all". Good catch.

That's looking at it backwards---this is a non parametric approach. The author didn't assume a priori that a point with a z score greater than N was anamolous, for some fixed N.

Instead, they determined N by looking at the past data.

(Contrast with the parametric approach of assuming normality and concluding that any point lying >3 sd from the mean is an anamoly due to properties of the gaussian distribution.)

I don’t think this is correct . You can calibrate a threshold for a metric empirically without interpreting the metric probabilistically. Ie that z > N is empirically a good threshold for anomaly detection calls for no necessary further interpretation.

Sure, but for most processes that’ll yield disappointing performance.

Any tool can be used without regard for the consequences, but knowing a tool’s statistical properties yields know-how about the consequences of its application. It’s often a matter of the costs of error / stakes of the most problem at hand. Cheap solutions are often the best for cheap problems.

Reply on news.ycombinator.com