When Your Profiler Lies | Hacker News Reader

When Your Profiler Lies

randomascii.wordpress.com

94 pointsnikbackm14 comments

Is the laptop using the HPET for the QPC timer? (10MHz..) and the added latency[0] of calling into that is messing up the reported timings in the profiler?

[0]https://docs.microsoft.com/en-gb/windows/desktop/SysInfo/acq...

I haven't checked. My workstation has a 2.53 MHz QPC that is expensive to call and (after stepping in to the function) I found that:

1) The 2.53 MHz QPC is just the rdtsc frequency divided by 1,024 2) The slowdown is from calling ReadTimeStampCounterFromEmulator, which it seems to discard the results of

My laptop doesn't run at 10.24 GHz so it probably is using HPET rather than rdtsc/1024. This makes QPC more expensive but I don't think it is messing up the reported timings - events are just outright being lost, as far as I can tell

'Why is QueryPerformanceCounter running at 10 MHz on my Lenovo P51 laptop?' Wild guess, Spectre mitigation ?

Huh, you may be right - in JSC and webkit the timer precision is deliberately limited for precisely that reason.

Native code that wants more precision can use rdtsc. JavaScript in the browser should have its timer precision reduced, to prevent timing attacks, but that should be done by the browser, not the OS.

The reduced precision isn't what really bothers me though - it's the cost. My measurements suggest that QPC is about 25x as expensive as rdtsc on this laptop - expensive enough to make calling it too frequently a problem.

This article explains some of the reasons why rdtsc might not be used to implement QPC, and I'm curious as to which one applies, and how machines this applies to:

https://docs.microsoft.com/en-us/windows/desktop/SysInfo/acq...

It would probably be a lot easier to deal with such problems if Windows was open source and you could just have a look what the system libraries are actually doing.

We might cry though.

Not necessarily. There are many bugs in open source and having the access to the source doesn‘t mean it‘s easy (as in, it doesn’t take at least a week) to localize the bug. If you claim it is I have a task or two performance issues for you, in the open-sourced Firefox. Thr issues exist for years, and the cause(s) is(are) still not narrowed down.

I found bug in nanomsg: https://github.com/nanomsg/nanomsg/issues/1005 . Still cannot find what causes it.

Really it doesn’t.

Sure there are some times where having source access to other projects helps, but in most cases it really doesn’t.

Essentially at best it turns “this function takes more time than I would expect” into the same thing with “... I think because of X”.

You may think “I could fix it” but again in most cases any fix will require you knowing why certain trade offs were made. Designing around internal design also means you may end up having overly complex code that continues to exist if the library changes design to improve that case. It may even become slower, or worse if your software is important enough may prevent future improvements that would benefit everyone.

So no, being open source does not magically make all bugs trivially fixable.

> being open source does not magically make all bugs trivially fixable.

Nobody claimed this.

EDIT: What is being claimed is that buggy code being open source helps the debugging experience, which is a weaker claim than what you tried to argue against.

Not easy, but I agree that it would make it easier. I would have to have stepped through the process shutdown code in a debugger, walking through the code to understand when the lock was acquired, and maybe setting breakpoints to count how often it was acquired.

Then again, that would require kernel debugging and a second machine and there would be numerous logistical problems, so I might not have bothered.

I spent too much time trying to reconcile the discrepancy and I ultimately failed - maybe source code would have helped.

I used to work there and could step into windows source when a win32 function wasn't acting how I expected. The bug was almost always in my own code of course, but seeing exactly what was causing the error was extremely helpful in fixing the problem.

Open sourcing (or at least public sourcing) the windows userland would make development on the platform a lot easier.

No, not easy, just possible.

Reply on news.ycombinator.com