I like how the article talks about the uselessness of percentiles, when a lot of people are still using averages!

Seriously, one of the best tool we use is to count thresholds and buckets. On a per-route basis, identify a threshold that you deem worrisome (say, 50ms), and then simply count the number of responses above that. Similarly, create buckets, 0-9ms, 10-19, 20-29... I say "best" because it's also very easy to implement (even against streaming data) and doesn't take a lot of memory. Not sure if there's a better way, but we sample our percentiles, which makes me uneasy (but keeps us to a fixed amount of memory)