95% latency seems way more useful as a metric than max() would be. It’s better to spend most of your time improving the experience of most users rather than debugging ultra-rare issues that rarely affect users.
So the user experience of your system is strongly driven by the tails of the latency distribution. Practically all of the useful information is beyond 95 %, not below it.
You have received several good responses already, and I'll add just one more: the latency distribution of many software systems is in practise closer to a power law than a normal distribution. This means the "outliers" dominate almost all ways to look at the distribution, which means you will find the very high percentiles (including max) to be more informative of what going on (on a per-request basis) than the lower percentiles.
Another feature of power laws is that it's hard to estimate their shape based on the central values -- one of the more effective methods is to MLE fit a tail exponent to the tail (i.e. what's beyond the 95th percentile) and then extrapolate from that also to central values when necessary. This yields a less biased estimate of the mean, for example.
For single requests, targeting 95% would probably be just fine back in the days of static sites served by a single Apache instance, but that doesn't describe many modern systems. Nowadays, a single HTTP request to a load balancer can trigger a flurry of blocking queries to fulfil it. (eg a distributed ElasticSearch query that can't combine the results from each node and return until EVERY node has responded) then your worst-case performance will quickly begin to dominate.
You need a lot more "9"
In practice, what you really want to focus on is solving the modes of your system but identifying them can be quite hard (aside from the obvious one of max). %-iles are a useful tool but don’t mistake your tool for the truth or accidentally mistake the measure for the target.
Working on ultra low latency systems for algorithmic trading I can tell you we actually care and hunt down maxes more than the 95th percentile.
They tend to be much bigher outliers in HFT and the transactions you typically loose money on.
Most of the time, you will experience adequate performance from our service. However, from time to time, you will also experience anomalies. We won't be fixing the sources of anomalous behavior because they're rare.