A portable high-resolution timestamp in C++ | Hacker News Reader

A portable high-resolution timestamp in C++

blogea.bureau14.fr

45 pointsshin_lao43 comments

If you want to see what a rabbit hole looks like, parse this sentence from the article :

"The beauty of this is that with a precise enough timer, you also solve the multithreading issue because nothing ever happens at exactly the same time."

In the first part the author expresses a technique which narrows the window for failure, and adds a fallacy which sounds good but isn't true.

This way of thinking (narrowing windows to the point where they are probabilistically rare "enough") has been the source of many bugs.

Urs Hoezle (VP at Google) once said something I really liked which was "At a large enough scale, statistically impossible things happen every day." It is painful to accept but I've seen it in action.

Another quote echoing the exact same sentiment: "One in a million is next Tuesday"

http://blogs.msdn.com/b/larryosterman/archive/2004/03/30/104...

He'd be better off getting the time in seconds, and then choosing a random number as well. At least then the chance of collision would be a known probability!

You can have a different constant in every server, for example nanoseconds in server A will always be 0001, in computer B it will always be 0002 and so on. It's cheap computationally and there can't be no collision.

I had the same feeling when people suggested using completely random UUIDs for things like cookies and transactions.

This just comes down to probability. It is extremely improbable to the point that the heat death of the universe would probably happen first, yet it is something that can worry...

You're most likely to have a bug caused by a hardware failure than a problem caused by colliding UUIDs (provided they are properly generated or large enough).

Thanks for the feedback, but did you notice this part?

In our case, we have some additional homework to make it work on several nodes that is well beyond the scope of this post

As for multithreading, the functions are guaranteed monotonic.

"nothing happens at the same time" is a reference to the laws of physics.

I don't understand the use of clock_getres() here. A second of posix monotonic clock time means a second of real time, the clock resolution has no bearing on that. Seems to me that on any system where the claimed CLOCK_MONOTONIC resolution isn't 1ns this code will advance the clock at the wrong pace.

> This is not a hard task. Nothing we've done above requires more than reading the documentation carefully. Attention to details like this is what makes the difference between working and rock-solid software. #frencharrogance

Um, right...

This really looks like a problem with the Windows implementation of std::chrono::steady_clock more than anything else. This clock is nano-second resolution on most *nix platforms (incl. Mac OS X) and offers the strictly increasing guarantee required (without possibility of discontinuities).

It seems like (in the long term) it would be better to push some std::chrono::steady_clock Windows patches to stdlibc++/libc++/MSVC and use this instead of re-inventing the wheel.

Spot on: http://connect.microsoft.com/VisualStudio/feedback/details/7... and https://connect.microsoft.com/VisualStudio/feedback/details/..., and the last response:

Hi,

Thanks for reporting this bug. We've fixed it, and the fix will be available in the next major version of VC (i.e. after 2013).

steady_clock and high_resolution_clock are now synonyms powered by QueryPerformanceCounter() converted to nanoseconds. QPC meets the Standard's requirements for steadiness/monotonicity.

Additionally, the CRT's clock() has been reimplemented with QPC. While this improves precision and conformance to the C Standard (as QPC is monotonic), we are aware that this is not completely conformant. Our CRT maintainer has chosen to avoid having clock() return CPU time advancing faster than 1 second per physical second, which could silently break programs depending on the previous behavior.

So it is expected that VS2014 should have the fixes.

On a sidenote, I've seen most of the code, if not all, posted by the OP already before while searching for the same functionality (eg the piece after "you will write your own function:" is followed by something that to me seems like a straight up copy from some FOSS project, even the comments seems to match - ok I cannot be 100% certain on this but if it's the case it would be nice to mention the source). I can't find it atm, but I am sure a C++11 clock compatible implementation using QPC has been posted on StackOverflow.

Function you are looking for Windows is possibly not QueryPerformanceCounter(). It's unreliable when you consider various hardware, especially in multithreaded applications on multi-core/CPU systems. Even more if you use Windows under VM. QPC can use RDTSC(P), but it's only one of its options, and even if RDTSC(P) is used it doesn't mean anything reassuring actually.

Go with timeGetTime(), remembering about calling timeBeginPeriod(1) early (usually at the beginning of application) to set minimum resolution for periodic timers to 1 ms (well, it will happen only if HW provides that much resolution), and calling timeEndPeriod(1) after you stopped working with time (usually at the end of application). Milliseconds don't give you high-resolution, but at least working in this resolution is reliable. Having us or ns garbage is hardly any better...

In recent years QueryPerformanceCounter is actually quite reliable since it's guaranteed not to change frequency during runtime.

timeGetTime() and company though operate at the highest frequency that application has specified, and as such can be quite a drain on portable power systems (laptop etc). So when one application calls timeBeginPeriod(1) it means the laptop needs to wake up more frequently and hence is less power efficient.

What does it mean "in recent years"? Did QPC become better (and Sleep()/WaitForSingleObject() saner, as we're touching time-related stuff) in XP before EOLing? Assuming having Vista+, does it really work reliably even when called from a thread bound to only one core on various (not necessarily latest) hardware? Have you played with it under VMs?

QPC was buggy and is still buggy for many users out there. After 15 years Microsoft is likely getting closer to sane behavior and maybe in latest Windows 8 (8.1) and Sever 2012 (2012 R2) it really works reliably in different setups and loads (haven't tested it yet). OTOH "getting better" by sole die out of setups, where it was working incorrectly, is not really getting any better.

Viewpoint is important here. If you build your own infrastructure, you choose your HW/SW stack, can thoroughly test QPC behavior, then if it seems you go with same setup for your servers or what not [1]. But if you write software for all Windows users out there, then you don't have that much luxury. QPC is simply not good enough. You can always try with building some logic that falls back from QPC to lower resolution functions whenever odd behavior is detected, but unless you have HW/SW instances that you could test it on, then there is (high?) possibility you won't do it right, so maybe you shouldn't do it in the first place?

1 ms resolution at best should be enough in most cases. Increasing timer interrupt frequency from default 64 Hz (15.6 ms resolution) to 1000 Hz (1 ms resolution) indeed affects system behavior and can harm battery life, but it may be necessary evil. If we're talking about high-resolution time, then 1 ms resolution is simply that kind of thing for Windows (as ridiculous it may sound) if we take reliability into account. E.g. games, your multimedia applications and so on are already using timeBeginPeriod(1). Good news is that reportedly since Windows 8 the harm on battery is much less.

[1] But if you really have control over HW/SW stack for your servers, then you'll quite likely happily avoid using Windows...

FTA: "Assuming servers are kept synchronized enough (the enough depending on your application), you may just solve the problem by acquiring time precisely enough."

How on earth are "sub-microsecond timers" supposed to be synchronized anywhere?

We have special hardware that syncs our servers to an atomic clock in Colorado. It's about +/- microseconds off. Not sure about sub microsecond tho. Another problem with this is you will run into clock drift and NTP time adjustment bugs. Honestly I stopped reading once he mentioned that time was going to be his secret sauce. There are just too many subtle issues with using that as your globally unique identifier.

If I'm reading the spec sheets correctly, the datacenter rubidium/gps clocks typically used advertise < 5x10^-9 (worst case) to < 5x10^-11 (best case) accuracy.

Which spec sheet? Would like to read more about these.

Something like this:

http://www.spectracomcorp.com/Desktopmodules/Bring2Mind/DMX/...

The units are a bit confusing but the accuracy still translates down to a handful of nanoseconds per month, I believe.

Precision Time Protocol: http://en.m.wikipedia.org/wiki/Precision_Time_Protocol

Or a GPS disciplined clock

OK but the point in the article is to reduce reliance on shared state. This doesn't work in a database.

This is very possible with high quality software, for example, see http://fsmlabs.com

I'm pretty sure rdtsc is/was broken on Intel core 2 duos, especially those with power saving feature (like laptops).

Not broken, per se. Rdtsc counts instruction cycles. It's not an accurate indication of time when the clock rate is variable (and in fact pausible).

I believe it is actually not monotonic, which was a requirement for this library. That's why you end up using the high resolution timer which has a 10mhz clock rate instead.

I ran into this problem trying to get better than 1us resolution for a daq system which was hooked up to custom GPS hardware which had better than 10ns resolution.

http://stackoverflow.com/a/19942784:

"So; to cut a long story short, if you want an accurate performance measurement you're mostly screwed. The best you can realistically hope for is an accurate time measurement; but only in some cases (e.g. when running on a single-CPU machine or "pinned" to a specific CPU; or when using RDTSCP on OSs that set it up properly as long as you detect and discard invalid values)."

Correct. They only made rdtsc constant-rate with Nehalem and later chips, IIRC.

There is a flag on linux in /proc/cpuinfo that tells if it supports a constant rdtsc

The problem exists on AMD CPUs, not Intels.

[...] for Intel Core Solo and Intel Core Duo processors [...]: the time-stamp counter increments at a constant rate. The specific processor configuration determines the behavior. Constant TSC behavior ensures that the duration of each clock tick is uniform and supports the use of the TSC as a wall clock timer even if the processor core changes frequency. This is the architectural behavior moving forward.

Intel Architectures software developer system programming manual 17.13

A minor correction: POSIX only mandates CLOCK_REALTIME and CLOCK_MONOTONIC is optional (POSIX Advanced Realtime Extensions). Linux provides both as well as CLOCK_MONOTONIC_RAW, and some BSD like FreeBSD provide CLOCK_MONOTONIC_PRECISE.

For OS X, use mach_absolute_time, described here: https://developer.apple.com/library/mac/qa/qa1398/_index.htm...

What is the reason for using std::uint64_t? Shouldn't uint64_t work? C99 types should be already included in C++.

uint64_t is a typedef defined in stdint.h. std::uint64_t is defined in the std namespace in <cstdint>

Yes, though which do you propose should be used? It's nice that the type is wrapped in the std namespace, but for something like a basic standardized type, seems like a waste. I think I'd rather include stdint.h.

You're writing C++ code, no need to regress back to C.

Yes, but I agree with the commenter above. Wrapping it in std seems weird. By your logic, one shouldn't use int and long too, or may be they should be wrapped in std?

If it bothers you so much, just use 'using namespace std;' or 'using std::uint64_t;'.

http://en.cppreference.com/w/cpp/language/namespace

Or just '#include <stdint.h>'. C++ is TIMTOWTDI too!

Yes, thank you (TIMTOWTDI)! I was just curious to see what other's used, and was sad to see this comment: "You're writing C++ code, no need to regress back to C." Really? Sure it's from C, but it's a part of C++.

Going the route `using std::uint64_t`, you'd have to do that for every basic type. Which is fine. I'd just use that in a general header that's included everywhere in a project (stdafx, or what have you). Though at that point it doesn't really matter which you pick.

I'd prefer them to be included in C++ as regular base types really, to be on equal terms with int, char and etc.

It's extremely useful for argument-dependant lookup[0]. If a and b are "uint64_t" and not "std::uint64_t" then "swap(a,b)" won't work in a generic fashion.

http://en.wikipedia.org/wiki/Argument-dependent_name_lookup

how about being portable to non-Intel platform?

Really, nothing new to see...

Here's our (Bloomberg) version of this. The bsls::TimeUtil component is used to implement the bsls::Stopwatch. It supports high-res timestamps on OSX/Windows as well as Solaris, AIX, and HP-UX (SPARC, POWER, IA64). It's a good base to start from and we'll continually tweak it for more performance as platforms change (I think there a few patches in the pipeline).

https://github.com/bloomberg/bde/blob/master/groups/bsl/bsls...

Component docs: http://bloomberg.github.io/bde/group__bsls__timeutil.html

Eh, what about not relying on timestamps at all in distributed systems? Use lamport or vector clocks if you want a notion of time in a distributed system. This article makes me more reluctant to consider quasardb.

Reply on news.ycombinator.com