Hacker News Reader

Comment by philwelch | Hacker News Reader

This is why it’s so important to measure p99 latency rather than maximum latency.

true, percentiles are important for telling you how your system is doing and how typical users experience it. However, I contend that you should also measure max latency, just that is is less important for understanding typical usage. But seeing your max go crazy can signal that there is something else in the stack that needs work. Silly example that I hit: max was consistently 2hr on something that p99'd at like 1min and p50'd at 2 seconds. Dug in. The 2hr was a default tcp timeout being tripped due to some connections not being properly returned to the pool. Fix the pool logic and max times came down into the few minute range. This also lowered p99.

I agree with this, don't exclude extremes because a blog post or comment said it was bad, they can still be useful sometimes.

For general purpose web development response times, I like to refer to p99 and p95 for general healthiness. P50 doesn't do much more than make me feel better. Max has its uses, it can be a good timeout gauge as the commenter above me pointed out. If the RATE at which max values occur changes, you may have something worth investigating.

Good points.

To my point though, you mention that you noticed a "consistent" max of 2hr. Meaning the consistency was the interesting part. You probably consider this issue to be more important than some other hypothetical issue that caused, say, literally a single request to have a 2hr latency on Tuesday.

In other words, if you monitored over a period that was long enough to establish a statistically significant p99.9 or even p99.99 and then measured those percentiles, you still would have not only caught this issue, but also distinguished it from the other hypothetical issue. Monitoring your maximum and worrying about consistently high maximums might be a more effective mechanism, but in principle, you still care more about top percentiles.

In sum: top percentiles are more important than the max; but monitoring the max can still be useful.

Just like monitoring the arithmetic average ain't completely useless either. It's just far from the most important metric.

This! Thank you for your service to category theory!

Comparisons like this are what is going to lead us into the next 100 years.

This user is not https://en.wikipedia.org/wiki/Philip_Welch

How do you know I’m not?

To confirm though, I am indeed not that or any other mathematician and I can’t claim any credit for any contribution to category theory. I’m simply a programmer who knows a little bit about a few things, no one special :)

Going to the website on your profile made it immediately clear, but even without that confirmation, there's a much higher probability of you being a random programmer vs. a specific older mathematician.

Well I suppose it's possible that your "about" line is completely fake, but in that case it sounds like you want people to go around thinking you're not him.

Reply on news.ycombinator.com