Key Mathematic Principles for Performance Testers

msdn.microsoft.com

46 points·chris-at··6 comments

This is a introduction to fundamental statistics, and not a very instructive one. For example, this is incorrect:

"By definition, one standard deviation is the amount of variance within a set of measurements that encompasses approximately the top 68 percent of all measurements in the data set; in other words, knowing the standard deviation of your data set tells you how densely the data points are clustered around the mean."

The 68% refers to a rule of thumb for normal distributions, not the definition of standard deviation.

https://en.wikipedia.org/wiki/Standard_deviation

Also, applying this rule will get you in big trouble if your data happens to have a mean that's less than zero:

“Data with a standard deviation greater than half of its mean should be treated as suspect. If the data is accurate, the phenomenon the data represents is not displaying a normal distribution pattern.”

Not only that, but saying "the top 68 percent" is flat out wrong. Even if a normal distribution is assumed, it should say "the middle 68 percent" or something to that effect.

Guide to Performance Testing Web Applications home page: https://msdn.microsoft.com/en-us/library/bb924375.aspx

under the "Medians" section: "In the case of the exemplar data sets, Data Sets A and B have median values of 4, and Data Set C has a median value of 1."

I didn't bother to check data sets A and B, but C is obviously wrong (Median for C would be 4).

Things like this are why MS are in the state they are in.

Please don't post unsubstantive dismissals to HN.

Reply on news.ycombinator.com