Falsehoods Programmers believe about Time
infiniteundo.com
infiniteundo.com
1. Time never goes backwards (as other people have pointed out, time zones break this).
2. UTC time never goes backwards (as other people have pointed out, leap seconds break this).
3. The system boot time never changes. On most platforms, the current time is defined as "boot time plus uptime", and setting the current time is performed by changing the boot time.
4. System uptime never goes backwards. Some platforms handle setting the current time by changing the system uptime.
5. POSIX's CLOCK_MONOTONIC never goes backwards. On some platforms and virtualization environments this can break with CPUs shared between virtual machines.
6. On systems without virtualization, CLOCK_MONOTONIC never goes backwards. On some platforms this can occur due to clock skew between CPUs.
> 6. On systems without virtualization, CLOCK_MONOTONIC never goes backwards. On some platforms this can occur due to clock skew between CPUs.
Could you explain these situations in more detail? Or cite a source I can take a look at?
CLOCK_MONOTONIC is what I use for timing quite often. I tend to do soft real time stuff and that clock seems the best suited for my tasks.
If your process is rescheduled to a different CPU, it must still go forwards regardless of TSC variance between the CPUs.
Of course if your uptime hits 68 years or so, the clock will wrap. If your app can't have any downtime in 68 years though I hope you've got the budget to think about this sort of thing :)
(6) is a case of "synchronization is really hard" combined with "benchmarks measure system performance, not system correctness". Most high-performance timing these days involves reading an on-die clock counter, scaling, and adding a base (boot time) value. For that to work on SMP, the clocks need to be synchronized -- and they don't start that way, since CPU #0 is enabled first and does some hardware probing before it turns the other CPUs on. Even worse, on many platforms, power-saving features will slow down the clock, resulting in the counters getting out of sync.
As alexs says, CLOCK_MONOTONIC should be monotonic... but in reality, it's much faster to return a mostly-good-enough value. In FreeBSD, in addition to CLOCK_{UPTIME, REALTIME, MONOTONIC}, we have CLOCK__{FAST, PRECISE} so that applications can choose between accuracy and performance.
CLOCK_MONOTONIC is what I use for timing quite often. I tend to do soft real time stuff and that clock seems the best suited for my tasks.*
As long as you avoid virtualization, turn off all power-saving features, and your "soft real time" can tolerate non-monotonicity on the sub-microsecond scale, you should be safe.
I have seen good programmers be surprised by the fact that POSIX time goes backwards in the event of leap seconds, as I was when I learned it. I think it would have at least as much punch if you edited your "falsehood 2" to be about UNIX time instead of UTC. As a bonus, you would also be correct ;)
http://statoids.com/tbr.html -- "Note that the government frequently changes its mind [about DST] at the last minute."
E.g. the "October revolution" falls in November - Russia didn't change until the Bolcheviks took power in 1917. And Greece didn't switch until 1923... China switched in 1912, but different factions in the civil war used different systems and it wasn't until 1929 they got a single (Gregorian) calendar again.. And there are many other countries that switched "recently".
But don't worry, they're just small countries.
Like Russia.
And China.
In the 5.4-ish time scale, there were a lot of clock related problems. One of them had to do with frequency scaling...
PCs have many time sources. The processor has it's own internal clock that ticks at a very fast rate (nanoseconds). There's the wallclock time which ticks at a slow rate (seconds). The internal clock starts at 0 when the system boots so it can't be used for wallclock time without adjustment.
Some Operating Systems (like Linux), get the boot time from the real time clock (slow tick rate) but then compute the current time by adding the CPU internal clock to it.
The CPU internal clock (TSC) can be wildly inaccurate for various reasons. One of them is frequency scaling which actually changes the frequency of the TSC dynamically. Unfortunately, if you're changing the frequency of the TSC on the host, guests that are running and accessing the TSC directly don't realize this has happened.
So if you scale the TSC frequency by 50%, time starts moving 50% more slowly. BIOS can also scale processor speed on some servers without the OS knowing which can lead to the same problem on bare metal.
More modern processors now have fixed TSC frequencies and KVM now has a paravirtual clock source both which address this problem.
BTW, Windows does not use the TSC as a time source so Windows typically won't have this problem (although it has other problems).
Time keeping in virtualization is fun :-)
37. Okay, quarter hours.
38. Okay, seconds, but it will be a consistent difference if we ignore DST.
40. You can wait for the clock to reach exactly HH:MM:SS by sampling once a second.
The best part was that this was in Javascript (and obviously he was not using a timeout). The entire page would lock up while it waited for the second to elapse. He never even figured out the obvious usability concern because he was so confounded by the fact that the procedure wasn't getting called after a second had passed.
Ok, to be fair, it was a freshman year programming class - not an unexpected or even unusual mistake. But it gave me a chuckle just now remembering it.
Another amusing anecdote: I got an old PowerBook out of storage and booted it up. The battery had run flat, so the clock reset to 1970. Spotlight noticed this and decided it had to reindex the entire drive. At some point during all of this, NTP kicked in and reset the clock to the correct year. I wanted to see how long Spotlight would take to finish indexing, so I popped down the menu. The progress bar indicated that it was about halfway done, and based on how long it had taken so far, it estimated that only another 40 years would be required to finish!
I just found out today about Calcutta time, which until 1948 was GMT+05:30:21, but I didn ’t realize it could be even worse than that.
N+1. OK, historical oddities aside, the offsets between two time zones won't change in the future.
N+2. Changes in the offsets between time zones will occur with plenty of advance notice.
N+3. Daylight savings time happens at the same time every year.
N+4. Daylight savings time happens at the same time in every time zone.
N+5. Daylight savings time always adjusts by an hour.
N+6. Months have either 28, 29, 30, or 31 days.
N+7. The day of the month always advances contiguously from N to either N+1 or 1, with no discontinuities.
Explanations:
(N)-(N+2): There exist enough jurisdictions in the world that time zone changes occur often enough to require regular updates to the time zone database, more frequently than many distribution release schedules occur.
(N+3)-(N+5): Jurisdictions change DST policies even more frequently than they change time zones.
(N+6)-(N+7): September 1752 had 19 days: 1, 2, 14, 15, ..., 29, 30.
In the British Empire, anyway.
As for "all modern computer systems", Unix time is completely ignorant about anything except seconds.
In hindsight, it seems like a brilliant decision.
https://en.wikipedia.org/wiki/POSIX_time#Encoding_time_as_a_...
That's not correct, actually: UNIX time tracks UTC instead of TAI meaning it "corrects" for leap seconds. As a result, UNIX time is not "the number of seconds since epoch" but "86400 * (number of whole days since epoch) + (number of seconds since midnight)", and UNIX time will go forwards (never so far) and backwards on leap seconds (a second will repeat in most implementations as the day goes from 23:59:60 to 00:00:00, as they have the same timestamp).
That's only in the British Empire. Other countries moved to the Gregorian Calendar at different times. It began being adopted on 15 October 1582 (the day after 4 October 1582) in some Roman Catholic countries (Spain, Portugal, Italy, Poland). Russia didn't switch until 1918.
So, you could also have:
N+8. There is only one calendar system in use at one time.
And the list hasn't actually covered the Gregorian calendar at all, since many people may not know the actual leap year rule.
N+9. There is a leap year every year divisible by 4.
The actual rule is that there is a leap year every year divisible by 4, except those divisible by 100, with a further exception to that exception for years divisible by 400, (so there is a leap year on such years). The year 2000 is one such year divisible by 400, so we have not yet passed a year divisible by 4 which is not a leap year since the invention of computers.
> 19. The system clock will never be set to a time that is in the distant past or the far future.
> 20. Time has no beginning and no end.
The reason this is impossible on Windows and downright awkward on Unix is because DST changes from time to time. Unless you have something like tzinfo, you cannot work out a particular date and time by merely adding a duration to another date and time in the past.
Instead, you must work out the timezone you are doing the calculation in, then work out whether the duration needs to incorporate a DST change, which involves working out what that DST changeover date and time was...
Of course, it's impossible on Windows because Microsoft only give the start and end datetime offsets for the current year. Amazingly, after all the patches they've had to issue to fix DST changes over the years, they still haven't implemented a Windows equivalent of tzinfo yet. And may never do so, even though they really aught to, given they are one of the leaders in calendaring software.
http://www.nytimes.com/2011/12/30/world/asia/samoa-to-skip-f...
"Aircraft software can be serious business" http://www.defenseindustrydaily.com/f22-squadron-shot-down-b...
"The [F22 Raptor IDL] post-incident report" http://it.slashdot.org/comments.pl?sid=224098&cid=181501...
"[undetected software bug in the] Northrop F-20 Tigershark laser inertial navigation system" http://www.f20a.com/f20ins.htm
TL;DR: calendaring is hard. really hard.
But really, with regards to time use whatever library came with you programming language.
That's after the Mayapocolypse.
On the other hand, requiring that clocks be set to within, say, five minutes (Kerberos 5) is a design consideration. This is why Kerberos is usually used with something like NTP.
But beyond this there are a lot of things you can't do (like expire cookies) if you don't assume that client and server have similar times on their clocks. Sometimes it makes sense to require things you can't assume to always be true.
Right. I love this. You can't assume it unless you document it as a requirement and make someone else make it true for all the systems your software runs on.
Never forget that specifications are a contract, an agreement entered into between the implementer and the user. If either side lets down their end, the agreement is void and the software can only fail. Either. Side.
Even if you have a mathematical proof that your software is correct, it's still only correct given certain assumptions taken as axioms in the proof. Violate those axioms and your software can't be held responsible.
"Beware of bugs in the above code; I have only proved it correct, not tried it." - knuth
You don't expose the underlying long value publicly so you end up with something like the following:
private long primitiveDate;
public Date getDate() { return new Date(primitiveDate); }
-----
This prevents calling methods from doing something stupid like:
o.getDate().setTime(123455L)
to modify a date that shouldn't be editable.
private Date date;
public Date getDate() { return new Date(date.getTime()); }
private final long creationTime = System.currentTimeMillis();
so it still has advantages over making the field a Date object.
In Unix-land, this data type and boatload of library functions is the operating system. The system provides and deals with local time conversion when necessary. If your application isn't very involved with time (eg. it's not a calendar or scheduling application) then it is sensible to use Unix timestamps.
This ties you to a Unix OS, which isn't usually too bad a decision since other more important things do as well. On the other hand, using your programming language or database ties you to that database or language, which is arguably worse.
Which can lead to problems as well. Basically when dealing with "local" time (DST), you need a reliable source of data and the question boils down to whether you want system administrators to keep OSes patched and up-to-date every time tzdata is updated, or whether that should be handled at the application layer.
Java chooses to bake in tzdata, and so do a number of other app-layer platforms. JS in the browser relies on the OS instead of bundling binary tzdata. Windows and a few commercial *nix platforms do not bundle tzdata either and maintain their own definitions. The crowd-sourced tzdata has demonstrated itself better than even what Microsoft ships in Windows. I don't know anyone who would claim a dataset other than tzdata is "better".
There might come a point in time where browsers have sufficiently advanced automatic patching invisible to the end user that they would be better off bundling tzdata internally rather than relying on the OS because they can guarantee their data is better in a higher percentage of cases. It all comes down to whose update system is more seamless and more likely to occur given all the external factors that come into play. (e.g., a user might have local machine privileges to apply browser updates, but not OS updates)
Also note, there are plenty of database systems that either have no proper datetime datatype, or have primarily datetimes that include no TZ info.
On Linux, if you run `info date` and go to "input date formats":
Our units of temporal measurement, from seconds on up to months,
are so complicated, asymmetrical and disjunctive so as to make
coherent mental reckoning in time all but impossible. Indeed, had
some tyrannical god contrived to enslave our minds to time, to
make it all but impossible for us to escape subjection to sodden
routines and unpleasant surprises, he could hardly have done
better than handing down our present system. It is like a set of
trapezoidal building blocks, with no vertical or horizontal
surfaces, like a language in which the simplest thought demands
ornate constructions, useless particles and lengthy
circumlocutions. Unlike the more successful patterns of language
and science, which enable us to face experience boldly or at least
level-headedly, our system of temporal calculation silently and
persistently encourages our terror of time.
... It is as though architects had to measure length in feet,
width in meters and height in ells; as though basic instruction
manuals demanded a knowledge of five different languages. It is
no wonder then that we often look into our own immediate past or
future, last Tuesday or a week from Sunday, with feelings of
helpless confusion. ...
-- Robert Grudin, `Time and the Art of Living'.1. Weeks start on Monday.
2. Days begin in the morning.
3. Re: 2, holidays span an integer number of whole days.
Explanations:
1: In Israel, the week starts on Sunday. Most programs have support for changing the "start of week day". Most programs.
2-3: In the Jewish calendar, the day starts when the moon comes out. This means that holidays that most calendars write as "Wednesday" will actually start on Tuesday night, and last until Wednesday night.
say what? afaik (speaking as a jew here), it's when the sun goes down. (moonrise, averaged over all time, happens at any point in the 24-hour cycle.)
possibly you're thinking of when (jewish calendar) months start?
Actually I just checked to make sure, it turns out the 3 stars thing is based on a definition by Maimonides: http://en.wikipedia.org/wiki/Jewish_calendar#Measurement_of_....
The Moon is not mentioned.
I got bitten by this once when developing a scheduling tool for project management. Since daylight-saving offset changes were always close to midnight, I assumed they would not occur while anyone was using my program (and there was no point in using it outside "office hours.")
Then a technician on a night shift used my program on the night DST kicked in, it went into an infinite loop and took down the CRM database server, disabling automatic software updates and an (unrelated) DRM system.
https://en.wikipedia.org/wiki/A_Deepness_in_the_Sky#Interste...
i just ran a quick test, and for a specific millisecond around now, about 13,000 distinct timestamps have the same string representation. if you use that string representation as a serialization of any one of those timestamps, it will always map back to a single float value, which will be only one of those 13,000, meaning the others aren't round-trippable.
the system implements a comparison tolerance for floating point numbers, but this helps only slightly, as only about 1100 of those 13,000 test as equal to the one you get if you enter it as a string.
the end result is that you can have data printing to the screen that you can't actually find in the system because its string representation doesn't match its internal one due to precision issues.
(the solution is not to use the type--they deprecated it in favor of one based on longs of nanos several years ago.)
You know who goes to the doctor a lot? Centegenarians.
assert(month(now()) == month(now() + 24.hours))
I can easily imagine somebody introducing a bug that relies on a 24 hour period ending in the same month as it began.
The title might better have been "Things programmers have tried to do with time."
A big issue is dealing with timezone conversions especially because different applications represent time zones with different english language versions of the names, like "US Mountain Standard Time" (used by Outlook) is the same as "US/Arizona" (used by PHP among others).
It's not quite that simple. During Daylight Saving Time, Arizona stays on US Mountain Standard Time; i.e., it's the same as Pacific Daylight Time and one hour behind Mountain Daylight Time. During the rest of the year, Arizona is on the same time as the rest of Mountain Standard Time, or one hour ahead of Pacific Time.
zoneinfo's form has the benefit of having a nice, unambiguous way to refer to the various daylight-saving time exceptions (arizona, indiana, hawaii, etc.).
recently i've seen huge timezone lists that basically throw in the kitchen sink--they'll have the entire zoneinfo set, the common american names, miscellaneous other common regional names (euro, australia, etc.) and raw whole-hour offsets as well. makes for a long drop-down to navigate....
Adobe managed to build a pretty shitty Date object for ActionScript that takes days as a 1-indexed parameter, but months as 0-indexed. Hopefully ActionScript is not used in banking [1]
[1] http://help.adobe.com/en_US/FlashPlatform/reference/actionsc...
I'm so guilty of that one.
But at least in the western system it shouldn't be possible to it to span years.
By noticing an automated test that failed 2 weeks out of the year.
If on the other hand it's just the test code that's flaky, usually such problems can be alleviated by refactoring such that one passes the time function as a parameter. This is in preference to hard-coding calls to the system time in test, which imho one should never do.
Once an arbitrary time function can be passed as a parameter, one can provide a mock or fake system clock for test purposes. Ideally one would still want to test under daylight savings' conditions. But at the least this approach leaves one in a place where one can test the common 24-hours-in-a-day case without having the tests spuriously fail two days out of every year.
1) You're (possibly) assuming that both timestamps come from the same machine. They could be from two machines (Server timestamps a transaction start, client timestamps the end.) Clocks are not accurate, so the delta time is not correct.
2) Time A is before a DST change forward or back, Time B is after. Delta would be wrong by +/- 1 hour (assuming all other factors are tracking with accuracy.
3) Both times are taken on the same machine, but far enough apart that clock drift plays a factor.
4) Both times are taken on the same machine, but an ntptimesync cron job kicked off in between them and adjusted the system clock.
$ ./timetool 1130647000
1130647000 = Sunday Oct 30, 2005 00:36 EDT
1130647600 = Sunday Oct 30, 2005 00:46 EDT
1130648200 = Sunday Oct 30, 2005 00:56 EDT
1130648800 = Sunday Oct 30, 2005 01:06 EDT
1130649400 = Sunday Oct 30, 2005 01:16 EDT
1130650000 = Sunday Oct 30, 2005 01:26 EDT
1130650600 = Sunday Oct 30, 2005 01:36 EDT
1130651200 = Sunday Oct 30, 2005 01:46 EDT
1130651800 = Sunday Oct 30, 2005 01:56 EDT
1130652400 = Sunday Oct 30, 2005 01:06 EST
1130653000 = Sunday Oct 30, 2005 01:16 EST
1130653600 = Sunday Oct 30, 2005 01:26 EST
1130654200 = Sunday Oct 30, 2005 01:36 EST
1130654800 = Sunday Oct 30, 2005 01:46 EST
1130655400 = Sunday Oct 30, 2005 01:56 EST
1130656000 = Sunday Oct 30, 2005 02:06 EST
1130656600 = Sunday Oct 30, 2005 02:16 EST
1130657200 = Sunday Oct 30, 2005 02:26 EST
$
However, leap seconds -- during which time passes, but the timestamps typically do not -- and numerical issues stemming from truncation and subtraction do have a systematic impact and reduce the accuracy of the difference. You can address the former for timestamps in the past by simply taking into account the leap seconds; you can address the latter by using higher resolution timestamps, ie. using millisecond timestamps if you need better-than-second-accuracy for the amount of time elapsed.
[0] http://www.unix.com/unix-dummies-questions-answers/18754-tim...
Using localtime in perl for example (a very common method to read the system time) does not return a timestamp, but returns a formatted string (see: http://perldoc.perl.org/functions/localtime.html) that could easily be thrown off by a DST changeover.
N. There are 60 seconds in every minute. N+1. UNIX timestamps always advance monotonically.
points in time are relative to some fixed point, and are (more or less) dimensionless. durations are not, and are (more or less) vectors. this has a couple consequences: durations are independent of epoch, while points aren't, and only certain types of math make sense with each.
basically, the only thing you can do with two points is subtract them (yielding a duration)--the rest of arithmetic (including addition) is meaningless. the only things you can do with a point and a duration is add or subtract them (yielding a point). you can't do anything at all with a point and a dimensionless scalar. the only things you can do with two durations is add or subtract them (yielding a duration) or divide them (yielding a scalar). the only things you can do with a duration and a scalar is multiply or divide them.
(personally i'd say that even the commutative operations shouldn't necessarily be commutative--i'd say point+duration->point, but duration+point->undefined--but that may be a bit too strict.)
as a quick rule of thumb, if your code would break if you changed epochs, it's already broken.
1. The year is 2012. In Thailand, the year is 2555. In Malaysia, the year is 1433.
2. But surely that is not used on computers and the web? http://www.railway.co.th/home/Default.asp?lenguage=Eng (look on the right side: "booking can be until : 18/8/2555"
The computer clock might not be.
If you are moving faster than the speed of light, you are perceiving whatever you are moving away from in reverse, because the light you are seeing was generated before when you left. (you are passing into light that is older than the light you started with). In terms of space-time, you would then be going backwards in time, if your point of reference for time was the Earth (and hint, given that we have all these nuanced time thingamajigs, it is)
http://en.wikipedia.org/wiki/Lamport_timestamps
http://en.wikipedia.org/wiki/Vector_clocks
That said, UTC timestamps since the epoch are one of the more straightforward ways of dealing with time, if you must sully your hands with the foul concept of time at all.
Definitely. Leave their rendering to the experts, but to store times and dates in any other way is indefensible.
Since we add leapseconds, and UTC includes those leap seconds, it's hard to know "time since the epoch". For example, unix/posix time is not the number of seconds since the epoch.
Can someone explain when this isn't true? Is he referring to leap seconds, or local timezone DST changes? Or something more interesting I'm failing to think of?
Other than that, I can only think of local DST changes you mentioned.
I'm looking at you Xcode...
I hit this problem once 10 years ago and learned the lesson ffs.
In all seriousness time-stamps are some of the most annoying things to compare ever.
If anyone has any idea about what they could possibly mean, I'd much appreciate it.
15.9.1.2 Day Number and Time within Day # Ⓣ
A given time value t belongs to day number
Day(t) = floor(t / msPerDay)
where the number of milliseconds per day is
msPerDay = 86400000
Every time library I've ever dealt with will have serious problems with at least one issue listed on the original article or in the comments here. JodaTime (on the JVM) is by far the best, but even they have problems, and are creating a new library to solve those.
Now they have N+1 problems.
Most applications need only three 'classes' to represent time:
1. A timestamp (ie. number of milliseconds since midnight on 1 January 1970 GMT)
2. A Gregorian Date (ie. three numbers representing day, month, year)
3. A TimeZone (to convert between 1 and 2)
In Java the first and third types are perfectly represented by java.util.Date and java.util.TimeZone. The second class can be represented by something like this: http://calendardate.sourceforge.net/
(Disclaimer: I wrote CalendarDate)
You know that's not unix time, right? Unix time doesn't include leap seconds.
The point of the above 'timestamp' is is to represent a specific instant in time, independent of any time zone or calendar system. You can use Unix time for this purpose.
Also during leapseconds, unix time does funny things, like 2 second long 'seconds'. or it goes backwards to reset etc.
That is almost always a better assumption than “I can write a better one”.
1) implementing their own DateAdd functionality in c# and forgetting about leap years.
2) thinking the app server and db server's time would never get out of sync.
Fml he was so bad I wanted to kill him almost every day and got paid quite a bit more than me
As someone living in +0930, programmers get this wrong way too often - Crittercism gets this wrong, so I don't know when bugs happened in my apps.
1. For any non-trivial project, it is possible to estimate development time and/or effort with a reasonable degree of accuracy.
If the day is one on which we either spring forward or fall back and the time the clock reads somewhere in the shift hour, then the clock will be right either 1 or 3 times that day. And it's different in different countries.
https://en.wikipedia.org/wiki/Daylight_saving_time#Procedure
Virtual machines can be screwed with comprehensively. So can non-virtual ones, come to that, and your software likely isn't in a position to tell that it's already seen 02:34:17 GMT 2013-05-25 five times already.
So what can you do about that? Bloody nothing. Absolutely nothing whatsoever. You're screwed. Everything you could do can be screwed with in ways your program can't defend against.
The lesson: Don't worry about bugs you can't fix. Refusing to play games you can't win will keep your hair in your head a lot better than knowing the intricacies of human timekeeping systems past and present.