Hacker News Reader

Comment by secondcoming | Hacker News Reader

Performance profiling on a laptop is largely pointless. There's too much stuff trying to conserve power by limiting performance.

View on HN

Are your users going to be running your application on laptops? Will they have the same "conserve power by limiting performance" going on? If so, that is _exactly_ the environment you want to do performance work in, generally speaking.

It's about having a consistent measurement baseline. Say you run your benchmark once, then thermal throttling kicks in, then you run it again, and it takes twice as long. Is your code actually slower now? Should I wait until the fan turns off before I run it again? That data is noisy and useless. Take your measurements on a server or desktop with sane thermals and a full-size fan.

If you speed things up by 10% on your server, they'll get 10% faster on your laptop as well.

Yes, you have to be very careful with measurements, I agree.

> If you speed things up by 10% on your server, they'll get 10% faster on your laptop as well.

Depends on the speedup and techniques to achieve it. For example, speeding things up via more parallelism can lead to wall-clock improvements on servers but not laptops, precisely because the latter just end up doing more thermal throttling....

Ideally, you want to measure both ideal hardware and actual-user-hardware; often speedups on one will not be visible on the other and vice versa.

generally speaking, the advantage of async io is strongest for high performance server applications, especially in regards to the cpu usage required relative to the amount of io stuff you can do. with that in mind, "users running your application on laptops" would not be the most common case.

Yes, if your app is a high performance server app, measure in that environment.

But user-facing apps (the sort people run on laptops, say) have async I/O as table stakes, really. It's not even about throughput or CPU cycles: it's about the fact that if you have I/O latency on any thread the user interacts with the user experience will be terrible.

Now in practice maybe that means "just make the I/O async, but the performance details of that don't really matter too much".

Anyway, the overall comment was about performance profiling in general, not just async I/O.

I have done benchmarking on a linux laptop. Once you disable turbo-boost you get quite consistent results for CPU-bound tasks at least.

You have to do so much more to be able to reliably measure events on the scale of nanos. You need to lock C-states, disable P-state driver, isolate CPUs, get rid of RCUs, affinitize your tasks, enable low-tick mode, skew hr ticks, make sure you use TSC clocksource, set the cpu governor, get rid of vmstat, set correct idle driver, disable audits, and watchdogs and much, much more.

If you want to instrument only a handful events, yes. But for microbenchmarks which you can run for many iterations to get min/max/stdev (such as the benchmarks in the article) it's much easier. Disabling turbo often is sufficient to lower the variance far enough that old and new code are clearly distinguishable.

It has nothing to do with instrumenting and everything to do with platform noise.

+1 This guy profiles

Reply on news.ycombinator.com