Hacker News Reader

Comment by PaulDavisThe1st | Hacker News Reader

I think it worth noting briefly that almost everything discussed in TFA concerns bandwidth (network, disk, other I/O, CPU) and not latency. That's understandable, because performance for a lot of people is about bandwidth. But there are some of us for whom bandwidth definitely takes a back seat, and you need a different set of tools for tuning latency in Linux.

View on HN

That’s a pretty reasonable point. It’s much easier to increase bandwidth than latency too[1] so often caring about latency can be important.

0. Another thing people may want to optimise for is performance per watt, but I won’t say much more about it.

1. There are cases where bandwidth optimisations are latency optimisations, eg if you can fit more of your processes onto one box, you are reducing the average distance between the processes and whatever they talk to and hence average latency

2. A very obvious thing to do when optimising for latency is increase bandwidth enough that the bandwidth doesn’t throttle you

3. I feel like mostly if you are aggressively optimising latency, there isn’t much Linux tuning to do. Maybe I’m wrong – I don’t really know much about this – but I think it’s mostly pinning to a core, running tickles, doing user space networking, and then hardwarey things like tuning page size, SMT, power-saving settings, and other things like choice of hardware.

[1] https://pdfs.semanticscholar.org/bce7/5f78d340cac32dccd8631f...

I would say throughput rather than bandwidth, although the meaning is similar, people have been discussing throughput vs latency for a long time.

Sometimes you can improve both, but often it's a tradeoff.

What are the tools and resources you would recommend with respect to latency tuning?

Latency of what particular system?

I'm sure there's lots of info on tuning linux servers, both for throughput and latency, so how about for desktop workstation use cases?

Not the person you asked, but generally you might want to look at "frame-based" profilers. These are typically used in video games, but the concept is general, and can apply to other applications. The "frame" could also be something like a request or transaction being processed. I like Tracy[1], myself.

Another latency metric that you'll see, often w/respect to web apps and microservices is "P99" and similar. This is the amount of time in which 99% of requests get their response. For a higher percentile, you get a better idea of worst-case performance.

[1] https://github.com/wolfpld/tracy

With the caution that if your goal is to reduce latency for the realtime properties of your system, chances are that you can turn any number of different knobs on your out of the box Linux distribution but it will not result in a satisfactory system.

Reply on news.ycombinator.com