Store epoch times as 64-bit floats
leebutterman.com
leebutterman.com
If you think microsecond precision is good enough for anybody, then you could consider using an integer where the LSB represents 1 microsecond. But don't use floating point time, unless your specific application cares more about time scale invariance than time translation invariance.
18446744073709551615
1686340765688397824
I fully disagree with using float64. It wastes bits on an exponent and you cannot safely use == to compare timestamps that have math done on them or have been converted to/from a string (including e.g. sent as part of a JSON).
You don't want to get into floating point, it gets all fuzzy and then when you need to change it, there's a bunch of code to change.
And integer representations have traps too. To pick on your particular example: "timestamps" are not the same thing as "times" and if your code predicated on that, it likely has bugs already with monotonicity and uniqueness assumptions.
And the benefit is that, as the article points out, you can have easy subsecond precision, something that is much harder (and pervasively unsupported by existing libraries) with fixed point math.
Should one go to the mattresses to defend either of these designs? No. But on the whole my heart agrees with the linked article. If you have to make a choice today, for new code, stick with a double.
Only because its timestamp (as in `Date.getTime`) is actually the number of milliseconds since the UNIX epoch assuming no leap seconds. You can't keep the current scheme in a nanosecond precision (2^31 * 10^9 ~= 2^61, so float64 will truncate lower bits). Python had the exact same issue and it now has a separate `time.time_ns()` function as opposed to the traditional `time.time()` function.
What integer system do you expect to work well with nanosecond time values? As you just showed, a 64 bit ns value will rollover in O(2^32) seconds, which is already universally considered to be insufficient for an epoch time.
Again, this is splitting hairs. In fact as I see it doubles would be preferable in in that regime, as while you'll be dancing at the edge of their representation capability, they at least won't blow trying to record dates in the 1420's or whatever. You can get as close as is practical without worrying about representation space.
Just use int64.
I'll go one step further. Time should be represented as unsigned integers, while time-differences should be represented as signed integers.
"10 minutes ago" is int64. June 9th 2023 should be unsigned integer64.
I was under impression unixtime is best for timestamps. But no reason to store something like a birthday using them; just do a string YYYY-MM-DD?
Timestamp with TimeZone (including DST + timezone offset) is a different type from timestamp without TimeZone (ie: normalized to GMT). Timestamp - Timestamp results in "interval".
Perhaps my method I was discussing doesn't work. But the overall idea that "Timestamp" needs to be a different type than "Interval", is pretty key IMO to handling date/times appropriately.
A time_t can handle intervals just fine.
time_t interval = t1 - t0;Unambiguously resolving "Timestamp at Eastern Time - Timestamp at Arizona Time" is not as simple as you might think. Postgresql handles this case.
Trying to make "time_t" handle all cases is a mistake. We need like 3 to 4 types at a minimum to handle all of the edge cases that occur in the real world. (And if anyone teaches me of any more complications, maybe we'll need even more types)
Do conversions to/from local time only as a first and last step.
It's the only sane way to do it. The D runtime library does that, and it has proven to be robust and sane. We're also recommending that users do the same thing with different character representations - do all calculation and processing in UTF-8. Other sets are translated upon input into UTF-8, and the UTF-8 is translated to other character sets only on output.
Cool.
Now how do you enforce that? Spoiler: have the local-time be reported as a TimestampWithTimeZone.
When you're doing calculations, make them all Timestamps.
When you're done, you can remember that you've converted from internal UTC-timestamps into the proper output format because you'll have a TimestampWithTimezone again.
-----------
We're programmers. We have problems that are simple and enforceable through compiler constructs called "The Type System".
Why have the human figure out if they have time-zone back-and-forth correctly or otherwise made a mistake (ex: adding code to the wrong "part of the process" and accidentally mucking with a TimestampWithTimeZone input/output variable rather than the internal Timestamp-UTC-only type variables?)
Just... leave the job of checking this entire process to the compiler. If your n00b coworker messes up, the type system catches it and we're all set.
UNIX timestamps have no timezones. They are just the number of actual seconds that have elapsed since a certain commonly-agreed time. You measure the number of seconds between two events just by subtracting their timestamps arithmetically, and don't need to do any parsing or interpreting of timezones. You can just say "this hacker in Russia breached the firewall 35 seconds after this dev committed code in the US" just by subtracting two numbers.
The time an airplane takes off, the time your firewall detected a security event, the time a cross-timezone meeting starts, these are all good to store as UNIX timestamps.
Birthdays are different because they are inherently imprecise; humans tend to celebrate their birthdays when it's a certain day of the month in their local time zone, with little to no regard for the time zone they were born in. Someone born in Singapore (UTC+8) and lives in Alaska (UTC-8) might still celebrate their 30th birthday on the month and day of their birthday in Alaska time even though their body might need another 16 hours before it has technically experienced 30 years of life.
If only. UNIX mishandling of leap seconds makes this all a illusion, in reality you need to have logic to handle them
So yes, you have to handle leap seconds when you're converting from the timestamp to/from the time scheme you are interested in, just as you have to handle leap years, etc. The UNIX timestamp is just a different way of marking time, and is blessed with not worrying at all about how that time relates to the actual motion of the Earth. That makes it easy and consistent for common operations. It is not mishandling anything. It just can't shield you from the complexity of the forms of timekeeping that make it a point to be synchronized with the Earth.
Nope. UNIX timestamp is days since epoch × 86400 + seconds since midnight. And that is the crux of the issue.
Are you perhaps confusing that with the standard algorithms for converting between Unix time and "human" time? Because what you say isn't wrong for that case, and this is a classic trap for new players:
"A Coordinated Universal Time name (specified in terms of seconds (tm_sec), minutes (tm_min), hours (tm_hour), days since January 1 of the year (tm_yday), and calendar year minus 1900 (tm_year)) is related to a time represented as seconds since the Epoch, according to the expression below.
If the year is <1970 or the value is negative, the relationship is undefined. If the year is >=1970 and the value is non-negative, the value is related to a Coordinated Universal Time name according to the C-language expression, where tm_sec, tm_min, tm_hour, tm_yday, and tm_year are all integer types:
tm_sec + tm_min60 + tm_hour3600 + tm_yday86400 + (tm_year-70)31536000 + ((tm_year-69)/4)86400 - ((tm_year-1)/100)86400 + ((tm_year+299)/400)*86400"
(Quotes from https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1...)
Moreover, if the units you are referring to are "seconds" then the statement "[t]he UNIX timestamp is literally the count of units past the epoch" is also wrong. If not, what are the units here?
> That's not true. Unix time is specifically defined as "A value that approximates the number of seconds that have elapsed since the Epoch."
It's missing 27 seconds that elapsed since the epoch. (Plus whatever happened with rubber seconds before 1972). Maybe that fits within the definition of approximate.
But if you need to record events that happen on a leap second or the second before a leap second, unixtime is unhelpful.
If you want to record a duration that includes a leapseconds, unixtime is unhelpful, although if the duration is long enough, the difference may be of no consequence.
Mail an atomic clock easy to west, and a different one west to east, and they'll drift different amounts. Put one in a plane, it will disagree with one on the ground. Put one on a mountain, it will disagree with one at sea level. Put one at the pole, it will differ with one on the equator.
There is little sense in claiming one time is exactly equal to another. Close to some tolerance is useful.
I'm struggling to think of a context where we basically want to create timestamps for things that happened in the past. Especially things that we want to dynamically localize rather than e.g. just store as a straight up timestamped string.
Nobody's really solved this, it has to be tackled anew for each case of "comparing dates from long ago".
Why artificially limit yourself to only representing times after the epoch? It's not like an extra bit of range is going to make or break anything.
Using i64 for nanoseconds with the same origin (1970-01-01), we could cover roughly 1677-2262. Using u64 for nanoseconds we could cover roughly 1970-2554, or more generally about 585 years of timestamps from wherever a new proposed origin would be. Likely we don't need nanoscale precision for times far enough in the past, so I guess "far enough" is the definition worth arguing over here.
The same story is present with 64 bit nanosecond based timestamps, albeit we're keeping the mid date ever so slightly in the past for ease of conversion. If dates in 2264+ are important then just use a 128 bit timestamp, it'll fix the problem for many multiples the age of the universe instead of causing issues now to gain a few hundred years of padding in the future. Another common approach is to keep u64 second based timestamps and save the nanoseconds as another integer. This comes at the cost of "only" solving the problem for hundreds of billions of years but being interchangeable with those using non-fractional timestamps.
For example, the unix epoch can safely be 1970 because the operating system did not exist prior to 1970, so there's no real-world situation in which you'd be time-stamping a system event or log entry prior to 1970.
That said, time_t is usually a signed type in C. There are several reasonable historical explanations for that though, including the need to use -1 for an error return, and the fact that unsigned ints weren't present in C until 1978.
The Unix epoch was never an issue being 1970 because they knew they were making it as signed, not because they thought Unix systems would never deal with e.g. files from before Unix was created.
As far as I've ever read, negative unix timestamp values generally were not used. Since -1 was used as an error value, you cannot safely determine whether that value represents an error vs a legitimate timestamp one second before the epoch.
I'm not familiar with if/how early versions of Unix were able to interact with non-Unix files/filesystems, but I doubt they would preserve metadata such as mtime if so.
mmm I don’t think that’s true.
- your computer
Alternatively, you’d need an int65.
> "10 minutes ago" is int64. June 9th 2023 should be unsigned integer64.
Won't you run into problems with trying to turn "ten minutes before 1970" into a time?
IOW, if you subtract two `time` types, the result must be a valid `time` type, so you can add/subtract to it. The result cannot be a `timediff` type.
Huh? Of course `time` - `time` gives a `timediff` type. You can add/subtract to it: you add/subtract timediffs, or you can add/subtract a timediff to/from a time. Why would you ever be directly adding times? What is 100 AD + 100 AD? 200 AD? Not if you pick a different 0-point for your calendar ...
But yeah started using int64 30 years ago and all the fuckedness went away.
Step forward the MC146818, used in the PC/AT and emulated by chipsets since, whose built-in daylight savings time change rule implemented in hardware was rendered obsolete in 1987.
Unrelated I think the lobbyists that got daylight savings time changed charged the BBQ manufacturers $50k.
> Why am I not using float64 time already??? [fp used ot be slow and was not standardized, but it got faster and after a while you could count on having IEEE-754]
This is not the reason.
Until IEEE-754 fp was terrible. As in buggy terrible. It took years of pain to learn how to get it right. And you can see this in the standard: it's a very short standard, super easy to understand, trivial, really...if you ignore the enormous amount explanations, corner cases, special cases, surprising symmetries, and other copious testament to the well-meaning Mistakes that Were Made in the years before. Intel provided a ton of trade secret understanding from the mistakes in the 8087, and UCB maths prof was intimately involved. It's an incredible achievement.
The history of the standard is quite exciting but has been covered on HN several times before.
time precision was decreasing with time since system was booted and after about 100h of operation this accumulated enough to system believe that incoming missile is already unreachable using existing calculations and caused it to not react at all
I think that is great example why changing precision depending on time might be very bad idea.
imagine that afer Y2038 that slight drop in precision would cause some critical system to fail in unexpected way
A 24 bit mantissa isn't enough for anybody. A 53-bit mantissa is enough unless you need +/- one nanosecond precision at 100 days. If you need that -- and some people do -- your application is too exotic to influence the standards that apply to everyone else.
Jokes aside, Unix Epoch shouls be replaced by Tai time on every computer.
I.e., use a 536-bit unsigned integer to store Planck time with one Planck time following the big bang as the epoch, and use the top three bits as a tetration height instead of an exponent, and you can get real far.
All the best,
For the past, it’s possible to workaround by saying “no leap seconds” but this means that 12PM 1000 years ago might be any time of the day.
But that workaround not gonna work for the future. For many use cases it’s important to be able to represent date-time for many years in the future. The currently used formats allow that easily, at the price of these unpredictable leap second adjustments.
It's the same reason why you should never use floating point numbers for account balances.
Not sure about classic BASIC; I don't really recall there being a specific time type, though I know later things like Visual Basic and etc would just use the VARIANT date type above with a 0 day component.
I like Windows' FILETIME which stores time as 100-nanosecond intervals since the beginning of the year 1601. C#'s DateTime is similar but uses an epoch of 0001 January 1.
I know JavaScript and others simply use the usual unix Epoch but with milliseconds rather than seconds, so time() * 1000.
int64 "nanoseconds since the epoch" seems to be emerging as the de facto standard.
used in Golang [0] and adopted by Python starting in 3.7 [1]
that PEP also spells out some of the problems with using a float64:
> The Python time.time() function returns the current time as a floating-point number which is usually a 64-bit binary floating-point number (in the IEEE 754 format).
> The problem is that the float type starts to lose nanoseconds after 104 days. Converting from nanoseconds (int) to seconds (float) and then back to nanoseconds (int) to check if conversions lose precision:
The problem being that OP kinda glosses over them.
I added a comment apologizing to software archaeologists on 12 October 287,396 C.E., when the precision of double timestamps (with 1970-01-01T00:00:00 UTC epoch) drops to 2 milliseconds, and the reviewer was fine after that.
* https://github.com/kdeldycke/awesome-falsehood#dates-and-tim...
Microsoft, for one example, has been modelling times as a 64-bit unsigned 100-nanosecond count since 1601 (proleptic-Gregorian proleptic-UTC) for about 30 years, now.
* https://learn.microsoft.com/en-us/windows/win32/api/minwinba...
Daniel J. Bernstein in the late 1990s chose a 0 point for an unsigned count so far back that it pre-dates most estimates of the point of the Big Bang.
* http://cr.yp.to/libtai/tai64.html
1970 is not the mandatory origin for every timescale. (Indeed, in the early years of Unix itself there wasn't even a stable origin for time.) It is not a valid reason for dismissing the idea of storing time as an unsigned integer.
It's also sad to note that the headlined page's first sentence has one of the very falsehoods that programmers believe about time in it.
IIRC the main reason floating point numbers weren’t used in system calls back in the 90-ties was that kernel code couldn’t use the CPUs floating point registers, or they would have to be saved at the entry point of each system call and restored before returning to user space. This was deemed too expensive.
It doesn't overflow until the year 4147 -- but since it's unsigned it can't represent times before 1970.
It's not designed to store times in the past, so that's not an issue for its intended use, but using it as a general time representation format could be problematic (say, if you want to represent a person's birth date).
(An aside: At a previous job, I worked on some code that would display a person's age given their stored birth date. The code didn't handle dates before 1900. I did a little research and found that that wouldn't be a problem, since everyone born before 1900 is dead. Ten years earlier, it would have been an issue.)
(in other words a terrible choice, though not for the reason above).
C already has (since the 2011 standard) `struct timespec`, a structure consisting of a `time_t` representing seconds since the epoch (typically 64 bits signed) and a `long` representing nanoseconds. It's admittedly awkward for arithmetic, but it doesn't overflow for another 292 billion years. It gives about 200 times better precision than 64-bit floating-point (for the interval from 2004 to 2038).
32-bit signed time_t is still a problem, particularly for embedded systems that are difficult to update. (A lot of such systems probably don't need to know the current time.)
A 64-bit signed integer representing nanoseconds since the epoch doesn't overflow until 2554.
Floating-point time is likely to be good enough in a lot of contexts -- but the fact that it's non-standard, unlike `time_t` and `struct timespec`, is a big disadvantage.
Y2038 is a non-problem, the representable range is about ±29000 years around the epoch. The extinction of the dinosaurs is not representable, but I think it’s OK in practice, only a small minority of software needs dates that far in the past or future.
Accuracy is 2.5 times better than FP64, and is equal over the complete range.
Disappointing to me how visually appealing this site is for how useless the words/idea themselves are.