Storing Times for Human Events
simonwillison.net
simonwillison.net
https://news.ycombinator.com/item?id=42259689 14 days ago
https://news.ycombinator.com/item?id=42281983 11 days ago
https://news.ycombinator.com/item?id=42358025 3 days ago
https://news.ycombinator.com/item?id=42364372 reportedly 3 days ago but it's the same HN item ID as this one (second chance pool maybe?)
It's barely 2 weeks old, is it worth reposting that often?
Semi-related, I remember submitting something I found interesting that had been posted iirc ~40 days ago, but had barely gotten any votes so I submitted it once more and i just get redirected to the old ID and the system refuses to create a new entry. Okay no problem, I'll post later^W^W forget about it, but that makes me wonder: how can 4 submissions in two weeks even physically happen?
We just got unpaginated threads after several decades so it'll be a while before the developer cooldown is up.
Maybe Hacker News needs to extend the time period during which it checks for duplicate URLs?
It appears to be so. You can hover your cursor over the relative time of the post to see an absolute date which does not change.
But seriously, yes, I think this is a very reasonable observation that's still surprisingly controversial and accordingly could benefit from more boosting.
If an event repeats every Sunday at 8pm, people expect that to stay at 8pm local time. Which means you need to know the original timezone the event was created in and make sure the interval is counted in local time, respecting DST. A `timestamptz` field is not enough for that, you need the additional `timezone` storage.
Another thing that I think is useful to mention is the new extensions to ISO8601 time formatting that allow you to include the actual time zone specifier, which is now finally being standardized:
https://tc39.es/proposal-temporal/docs/strings.html
2007-12-03T10:15:30+01:00[Europe/Paris]
> Because of this format's long-term industry adoption, it was chosen for use in ECMAScript Temporal for both input and output.> Although neither ISO-8601 nor RFC 3339 specifications currently use this syntax, it's on a standards track led by the IETF SEDATE working group which includes ECMAScript Temporal champions as well as other industry participants.
It is very important to use a high quality time library that forces you to think about these things.
Sometimes you really just want to express that something is going to happen at a certain time, regardless of timezone. If I set my alarm for 7:45 am, I need it to ring when the clock says 7:45 am, wherever in the world I happen to be.
This is how most human beings think about time - school starts at 8 am, work starts at 9 am, McDonald's serves breakfast until 10 am, my doctor's appointment is at 4pm... these are all things that remain constant regardless of whether daylight savings time starts or ends, whether our business changes locations to a different state, whether we are conquered by the Romans and our calendar changes... so they should be stored as such, without a timezone attached.
Then from there you can combine a "date" and a "time" and a "location / timezone" to form a "datetime", to figure out the actual technological instant something needs to happen, based on the user's location and/or the location the event is going to take place.
One of my toughest engineering projects was a system that supports “every Thursday at 3pm” and the participants can be in different timezones with different DST rules. I learned more about time math and how humans think about this stuff than I ever cared to know.
We ran into problems like "someone in Chicago is meeting with someone in Arizona at 10am every day." Worked great until daylight savings time, which Arizona does not observe.
I've learned way more about how to think about, represent, communicate, store, and translate times than I ever thought was possible, but there are tons of war stories of people who have climbed higher on these Mountains of Madness than I can see through the fog of war.
The biggest problem is that if even programmers - who know this is hard, and has a lot of corner cases - mess it up, expect your requirements to be very very wrong with regards to special cases.
Come up with a list of example cases and ask what the expected output should be; but not just the corner cases, the normal examples too. There's a good chance even basic requirements regarding timezone conversions have not been thought about, or are wrong. Like the earlier post said, humans really only think in local time and for short periods in the future or past. Any conversion whatsoever can end up being pretty unintuitive (even if it's right).
If it has to do with scheduling, BE EXPLICIT. Show both (or more) local times and dates and let the user pick a time for either location. Maps with pins can be a very helpful UI affordance
If I'm in Portales New Mexico and I drive to see my sibling in Muleshoe Texas at what my clock says is "8:30 AM", that'd take me approximately 40 minutes, allowing me to arrive 9:10AM where I can grab a breakfast sandwich at McDonald's before meeting my sibling for shuffleboard. Except whoops I crossed a timezone discontinuity there, so actually I will arrive at 10:10AM and McDonalds will have decided to stop serving breakfast even though I'm hungry for eggs and the clock in my car happens to read "9:10AM". This isn't some reason to "do away with timezones" because that'd still have the same problem w/r/t travel and different perspectives; if I jet to London from NYC then McDs will also not be serving breakfest when I am hungry for eggs and my (unadjusted since boarding the flight) ticking Timex on my watch reads 9:10am. It's not solvable because people are experiencing different "times" in different places. Either we abolish the concept of time and it's always "whatever number my spiritual essence dictates in this moment via gazing at feeble mechanical devices that I enjoy the aesthetic of" or we swing the other way and all humans record all time in UT1 and we treat the shifting light beyond our windows as a mere inconvenience. We've done the latter with computers because they demand such rigor, implying we'll never do the former and abolish time (it's too useful!), and humans won't all abandon being bound to the sun and its vibes, so here we are in a world where we try to please the fickle and unreliable humans, ourselves, with the cold machinery of our computers (may their flickering lights forever bless us, amen).
Thus, there is only "what's an easy way to represent time that humans won't hate too much?" For most everything, that's "store it in UTC/UT1 and then convert to nearest approximation of what the user wants based factors like 'where are they now?'". Every other approach has pretty big downsides or potential failure modes. For example, if we use the system you just mentioned with a "just time" is "to do anything with it you also have to store enough info to turn it into the other representations". Thus, questions like "how far away is new years eve for my friend across the world?" will be confidently incorrect unless you account for the sun being up here and the sun being down there (typically via the timezone concept). Storing in UT1 at least means you can give the user a time that obeys axioms even if the user has to do a little legwork to adapt their brain to it, which most folks who _depend_ on timekeeping are willing to do to make sure things don't go awry.
That's a time outside of a location which works for the humans involved but which is completely unresolvable by the "time at a location" computer paradigm.
People do stuff like this all the time and don't think about it and it's unresolvable and not consistent.
This property can be surprisingly hard to implement on Unix epoch based systems (just ask Apple), considering time zones, DST and everything!
I wonder if this is the reason that early RTC modules would actually count time in 24 hour or even calendar-based formats, as opposed to a simple binary counter? Examples include some Game Boy cartridges [1], and I believe also Palm OS handhelds (I think they triggered an alarm once per day to recalculate the date, although I can't find a source right now).
[1] https://gbdev.io/pandocs/MBC3.html#the-clock-counter-registe...
https://www.youtube.com/watch?v=pr6HTiWrMmk (CuriousMarc)
https://tc39.es/proposal-temporal/docs/plaintime.html
> A Temporal.PlainTime represents a wall-clock time, with a precision in nanoseconds, and without any time zone.
> Temporal.PlainTime refers to a time with no associated calendar date
Personally, I work with a few that are missing "date" too, so at least it's symmetric. But I have no idea why no tooling has support for time data. It's such a common need, it doesn't make sense.
Oh I desire more from my alarms.
Ideally I would be able to set an alarm for 7:45am and mark it "all timezones by then-local clock" ... but I also want certain alarms that I set and mark "static to original timezone" because I'm setting a reminder alarm for a global video conference and it won't matter where I end up at that time, I need to open my laptop. ... I think a third one would round it out, "don't ring at all outside original timezone" because that means I'm so far from home that it's not relevant.
In the states, I think most people I know are familiar with "eastern time" or "pacific time." But I doubt that many of my friends recognize "Americas/New_York" or appreciate why they need to select that instead of "Americas/Indianapolis" when referencing eastern time[1]. I certainly wasn't familiar with referencing timezones this way until I had a job that cared a whole lot about timezones and had offices in several countries!
I like the idea of solving this by just asking for the location of an event.
[1] having lived in indiana a while ago, I'm assuming that Indianapolis gets its own TZ entry because it wasn't on daylight savings back in the 90s (?). Which was pretty annoying.
So, yeah, you either use location, present major nearby cities, etc. to get around the fact that the ordinary person has no idea what the identifier of their timezone is.
(We of course, both know why! And if you haven't seen the names before but you're a programmer, youprobably still make some assumptions about edge cases, precision, etc. But I'm assuming that is not the experience of the typical Google Calendar user?)
One issue with this phrasing is that a user might react--quite reasonably--with frustration like: "Gawd, stop asking me you stupid computer, it doesn't matter, I already told you it's an online-only teleconference, it's not really anywhere!"
I'm not sure about the best way to phrase it, but ideally we want the user to be answering: "Which clocks determine when the meeting starts?"
On the day DST starts, that event cannot exist, at 01:59 clocks will jump forward to 03:00. Vice versa, on the day DST ends, any time between 02:00 and 02:59 will happen twice.
These days, you could have a global company where some emergency occurs and you have to call a meeting at sometime on Saturday wrt your location timezone, but one of the other locations is several hours ahead of your timezone such that the meeting occurs during/around the transition period for their location's timezone.
So, not common at all. But it may happen. That's why they choose the transition times to occur when most of the LOCAL population DOES NOT schedule meetings.
I don't think there were any Saturday/Sunday virtual meetings when this timezone stuff was created... Simpler times...
You might not like DST, but leaving it in the hands of every entity to interpret is a far worse situation.
Imagine if there was a powerful body representing the population that could decree a change like this!
Instead, we only have a body powerful enough to decree that all of those people change their clocks on the same day.
Anyway, the best part of changing the hours instead of the clock is that not everybody needs to do it on the same day.
Don't scapegoat it too much: The underlying problem of time-shifts will still exist, and that edge-case still needs to be handled by other means.
DST just makes shifts more frequent and obvious. One might even say that it draws an important amount of attention to a broader class of problems.
This behavior was changed in a more recent edition of the SQL standard. Now, we have `TIMESTAMP WITH LOCAL TIME ZONE` which works the way `TIMESTAMP WITH TIME ZONE` used to work: basically just converting everything to/form UTC when it's stored/retrieved. The semantics for `TIMESTAMP WITH TIME ZONE` are now supposed to work more like the author's suggested approach: just gluing a time zone name to a regular old timestamp.
I'm not sure when Postgres will adopt the newer type semantics. Oracle has for a while now: https://docs.oracle.com/en/database/oracle/oracle-database/1...
Postgres has had that exact semantics for ages, and the documentation explicitly tells you in very large text that it only exists for compatibility reasons and that you shouldn't use it.
The correct semantics for "timestamp with local time zone" is the naive one, that you can only recover using strings nowadays, thanks to the SQL standard braindead decision. But the good (?) news is that if you have a problem that requires a database, it's very likely that you will need to handle time zones anyway, so it's not a big issue.
A cursory look through Calendar didn’t turn up any way to enter time zone independent events.
But Google Calendar also had the time for the departure and reurn flights blocked out (including departure airport/arrival airport both ways), yet it was too dumb to notice I was setting an appt for 8PM for a Las Vegas location - but Google Calendar decided to put an Eastern USA timezone on the appt.
That's how you know the Google Calendar devs/PMs don't travel often. And also none of the same Google devs/PMS cared enough nor high-level Google execs themselves use Google Calendar - cuz if they had the same thing happen to them, you'd know that it would have been fixed before the next performance review.
For instances like “bake my bread for one hour” you care about elapsed time/duration which is not subject to the whims of civil time as you are coordinating with the laws of physics, or gods of cooking fickle as they may be, so a absolute second offset such as TAI should be used.
And for the past, the unambiguous correct answer is TAI or other absolute second offset. Period.
This is because the exact time of occurrence of a past event is, ignoring relativity, fixed and corresponds to a specific second/time. This is completely unambiguous and can be converted, entirely losslessly and roundtripped, to whatever the corresponding civil time would have been in any timekeeping system you so desire. But again, this applies to the past only. The mapping between seconds to civil time and vice versa is subject to the whims of your government until then.
Just like with any other kind of data, store whatever you are semantically representing. If you are representing an absolute point in time (bread timer), store a Unix timestamp. If you are representing time components, AKA relative time, AKA human time, AKA calendar/clock/tz time, then store those components.
Also, most past clock times we care to store start out as future times.
It will just convert to a different, now agreed upon, civil time according to the calendar chosen. It is no different from citing the current second according to the Gregorian calendar versus the Mayan calendar. They both refer to the same “time”, but use different civil dates. The underlying thing that is convertible to different calendar formats is the real thing and corresponds to the absolute time of the past event.
edit: And no, it is almost guaranteed that most past time events are timestamps of the “current” time which has the same qualities as “past” time. Only future civil times should not use absolute second offset times.
What you do need to worry about is the relationship being after an event has been planned and before the event occurs.
I've been working on an end-to-end encrypted event invitation service and I chose to ignore timezones completely and just have a date picker and free-form text field for time. If users want to export the event as an ical file I just set it as an all-day event and put the time in the body of the event. It's not perfect, but it's usable enough for most people.
The one exception here is importing to Google Calendar which for some awful reason imports all day events with no time zone as 24 hour long events from 12am to 12am UTC the following day.
(I have a silly beta version of the app up at https://pinvite.app if anyone cares to try it out)
https://news.ycombinator.com/item?id=19500640
The problem is that, except for UTC, any timestamp relating to a user’s intention must ALSO store a geolocation. Dinner at 6pm? Great! Where (on earth)?
And I’m not sure what to make of storing a TZ qualified timestamp AND a non-TZ qualified timestamp. I’m not sure what problem that solves or how to reconcile if they differ.
I’m still in the “store as UTC” and be very careful about disambiguating at creation time.
Inb4 that other blog post that explains why getting rid of timezones is troublesome in everyday life.
Of course if we go down that path we should also add a format to indicate offsets. If I say I'll be back in 10 minutes, that interval shouldn't change no matter what happens to the time zone. To perfectly disambiguate it you would need to store it as current time (with timezone or UTC) plus the intended offset. This would also "solve" a lot of issues around leap seconds and the inability to predict them more than six months into the future.
Not really? There's not that many timezones, globally, and I think their polygon data is in OSM / could be obtained from OSM. Simply brute-force colliding a single point with all of them I would expect to be basically "instantaneous", but there are a number of trivial optimizations, like bounding boxes, that you could do to speed it up.
(Point-in-polygon is not terribly expensive; it's O(segments) in your polygon.)
That said, there are problems with that, as you mention. I just don't think "computationally expensive" is one of them.
Some users may not know that their Timezone is America/Los_Angeles but they should hopefully know where their venue is.
Some users may not know that their Timezone is America/Los_Angeles but they should hopefully know where their venue is.
You always treat the local time - 6pm on 5th December in Boise, Idaho - as the core truth and derive the UTC version from that. Then if Idaho decide to cancel daylight savings time you can update your calculated and stored UTC timestamps to the correct values.
> use that location to determine the correct timezone
Does this mean consult an "address to timezone lookup service?" Not sure if they exist, and I don't see another way without user specifying manually. Maybe a string match, but it could be wrong/unsure somewhat often.
> Then if Idaho decide to cancel daylight savings time you can update
Looking up events within a time interval is easy enough, those in city (timezone?) limits not so much. Consult a GIS system, search within polygons??? String match or every city in your database?
Or you can get a latitude/longitude and derive the timezone from that - I built an API for that here https://timezones.datasette.io/timezones/by_point?longitude=...
I wouldn't recommend it though: it needs a beefy machine and applying updates on an ongoing basis is fiddly. Better to use a commercial option and have that as an escape hatch.
If the context is an event, what does midnight mean? A one-day event might start at 6pm but not end until 2 or 3am. A computer considers that a two-day event, but that's kind of intuitively wrong. To a human, it's only a 1 day, and it doesn't end tomorrow. It ends, late tonight.
What if it starts Saturday at 9pm and goes until 6am? You would still consider that a Saturday event, but ends at sunrise, or early morning.
And how do we automate getting to know about, and performing, those updates in our application databases? If that is not easily automatable, I think storing the UTC time as well will cause problems.
Maybe the UTC time should be a calculated column?
For automation: install the latest tz database (I use pytz for this, usually) and then run a batch job to see what's changed. Not trivial but not impossible either. Running events websites is hard!
Super reliable.