Time Zone Bugs I Ran Into
blog.davidojeda.dev
blog.davidojeda.dev
Always store the authoritative version of a time, which usually is UTC for points in time, but local time for calendar based things.
The duplicate/missing hour around DST switching can also be quite annoying.
For anything interacting with planning though, the user's needs should be considered. Often what you actually want to store is a "recurrence rule" for a single event, while being sure to include the defining aspects picked, or at least confirmed, by the user.
>What time is it?
2am.
>It’s been 2am for 2 hours!
That shouldn't be a problem for datetimes in the past. tzdata contains not only the current offset but all historical ones as well.
It might be a problem for /future/ datetimes. If I say the meeting is at 1700 America/New_York and the local civil authority changes the offset in the meantime I probably don't want the meeting to move in local time.
I filed a bug (JDK-8214243) but basically it's a known issue and there's no fix.
Oh yes. I used to work for company which produced medical software which continuously recorded patient data. Those duplicated or missing hours were always the worst and error-prone use cases.
Don't know if they've fixed it but when I had a Fitbit, they never was able to get these DST change over events handled correctly. What's worse is that their website and their app both handled the change over in different ways, both wrong.
1 AM, 2 AM (CET), 2 AM (CEST), 3 AM, 4 AM...
or
1 AM, 2 AM (CEST), 4 AM (CET), 5 AM, ...
Then there's the possibility of timezones straight moving around which can make entire days replay or disappear e.g. in Samoa there is no 2011-12-30, the day literally doesn't exist, Samoa local calendar goes straight 2011-12-29 to 2011-12-31, because at midnight the country switched from UTC-11:00 to UTC+13:00.
If a night loses 1 hour, the train stops for less time at the scheduled stops, or arrives with a delay. If the night gains an hour, the train waits somewhere for an hour. Which sounds like a painful thing, having to travel (or wait) an extra hour because of clocks.
Interestingly if the night loses an hour, the trains are automatically an hour behind schedule...
I work for one of the largest banks in the world and I get to see a lot of code for some mundane backend stuff. For some reasons, people think that the day has always 23 hour, 59 minutes and 59 seconds and that there is a one second delay between one day ending and another starting. Then they are incredulous some trades have vanished.
There was a serious movement to stop data flowing around midnight until I reminded everybody that midnight happens 24 times a day (and not even that, there are fractional timezones, DST, etc. to complicate matters).
I thought this meant to search the logs / database and I was absolutely horrified that maybe I didn't understand how time works.
I think you're talking about hardcoding 23:59:59 as EOD in the source code though, right?
More surprising is that a number of expensive, precision, commercial GPS chips do NOT all handle leap seconds the same way. I worked in a shop that used about three different ones for their PPS and clock outputs and we got about three different behaviors from them. Evidently the leap seconds are broadcast in supplementary data and then applied on chip when it produces a UTC output.
Everybody just smears it over some period of time.
That is, give me events where time >=30/1/2020 00:00:00 and time < 31/1/2020 00:00:00.
You know what the problem is? The developer wants to have events (like trades, transactions, log lines, users signed up, etc.) from a particular day and for some reason they fear/dislike having the next day. For some reason the 31/1/2020 00:00:00 is bad so the will prefer 30/1/2020 23:59:59.
I honestly don't know where this is coming from but it is real and I spend literally tens of hours trying to explain this to people, sometimes with no avail.
Actually, I know where this is coming from. This is mistaken attempt at addressing the last second.
How do you design system that filters stuff by giving range of dates? Go to any reservation system and you will see that to select just today, you need to select from=today and to=today instead of from=today and to=tomorrow.
I think fundamentally, this is mental model that causes people to think about the time not as a continuum but rather in chunks of some unit.
Depending on the type of event they will select best-matching unit (typically year, day, minute or second) and treat it as atomic with the timestamp of beginning of the time duration representing entire duration.
So in those minds, 23:59:59 represents entire last second of the day and there is no problem. 00:00:00 would represent the first second of the next day and that would be a problem.
Unfortunately, this is wrong mental model for things that happen on continuum of time or when somebody needs to block things using different units.
It only really works for things that are really managed in blocks of a given unit. For example, I can understand specifying accounting date range as from 1st to 31st of the month. This is right. Accounting date is no date at all, it is a label for a set of events that through some rules were associated with the label.
dateNext = sprintf(‘%s’, datetime(date, ‘format’, ‘uuuuMMdd’) + days(1));We introduced this change after frequent complaints about our invoices when we still rendered a month as "01.03.2020 00:00 - 01.04.2020 00:00", while people are fine with "01.03.2020 00:00 - 31.03.2020 23:59" despite the minor inaccuracy.
Apparently some countries at some points had (and sometimes, like Ireland [2], have) a "winter time" as opposed to a "summer time". In the other words their standard time takes place in summer and the "winter time" has a negative offset to the standard time. Technically this is not different from a "summer time" in other countries, but C `struct tm` has a `tm_isdst` flag that ideally should be true in winter for those countries.
tzdata made transition to support negative DST offsets in early 2018, but this caused unwanted software problems so the choise was made to support the "rearguard" format that doesn't have negative DST offsets. This was an one-off incident for most cases, but TZUpdater didn't support the more recent (and more correct) "vanguard" format. And yet users tend to feed the vanguard format to TZUpdater and thought tzdata was at fault. This annoyance continued until TZUpdater finally supported the rearguard format in late 2019 (I think).
[1] https://www.oracle.com/java/technologies/javase/tzupdater-re...
[2] https://en.wikipedia.org/wiki/Time_in_the_Republic_of_Irelan...
It is a full fledged web app because it is a web app for managing thousands of different blogs and doing dozens of little random features, each of which has their own little bit of JS.
E:
For comparison:
3.5k word article on Wordpress with comments and analytics, scripting enabled: https://i.imgur.com/nTlMxwL.png
TFA, 900 words, scripting enabled: https://i.imgur.com/wWRMPts.png
Welcome to 2020. A lot has changed. People still use phones to access Internet but not exactly the same way (you may want to get updated on this in the closest Apple store. No, not a store with apples either). Oh, and please, were a mask.
The page loaded instantaneously on my 1Gbps link. My phone on 4G loaded it fine too.
It is also very much possible that Internet Explorer 3 is not supported either.
Yes, this is a rant in reply to yours because some people have better things to do than to analyze DevTools outputs to say the 12654 bytes were loaded instead of 12341 over 50 calls instead of 27.
I am personally very, very found of the philosophy of https://motherfuckingwebsite.com/ and truly believe that this is how web sites with information should look like. But the ones that are readable on my devices are great too. And I do not give a shit about the volume because I do not have to.
And yes, this is horribly annoying to those who have 120x50 displays over a 6kbps link with 1 MB of capping.
Like I said - I like minimal web sites but the ones that works are fine as well. This one works, I would have never wondered about the weight without your comment.
If a site starts to load slowly or is not readable, I will just skip it.
Many blogs don’t exist as a way to produce an income and absolutely don’t have enough traffic to do so.
All data should be UTC; timezones only apply when the data is shown to the user, possibly on the client itself, which likely can automatically display the right time for a UTC date.
UTC time probably won't always give you the correct result, since timezones and DST aren't fixed.
Say, you want to store the time of your doctor's appointment months in advance.
Here it makes sense to store the time with a timezone - but not the timezone offset. This makes sense because no matter what the government decides about changing DST or moving timezone offsets alltogether - your doctor's appointment is going to be at 0900am at a specific geographic location and since you will physically be there with your watch synchronized to your doctor's.
But say you have an online appointment with your psychotherapist who lives in a completely different timezone (possibly you do not know which one) from you at some specific time in the future.
Here making the appointment via UTC makes sense - to meet up you need a globally synchronized clock.
The hard part: how do you know determine whether you can schedule these two one-hour appointments right after each other and not have a conflict ? I am not sure it's perfectly possible.
No it isn't.
* Europe postponed planned DST abolition due to COVID19: https://www.timeanddate.com/time/europe/eu-dst.html
* Argentina had a period of deciding each year what the DST dates would be until one year they just decided not to: https://en.m.wikipedia.org/wiki/Daylight_saving_time_in_Arge...
* Morocco and Yukon, Canada made updates to their DST policies earlier this year: https://mm.icann.org/pipermail/tz-announce/2020-April/000058...
Basically, UTC is mostly fine for recording past events (e.g. event logs though even then pre-1970 events can be problematic), but for any future event it’s very risky, because time zones « move around » sometimes with very little heads up (down to weeks if not days for DST) but a user working in a specific time one expects their stuff to be fixed in their time zones’ frame of reference.
But of course this indices the problem of ambiguity around DST switch & other backwards TZ movements.
But it's not enough for things agreed to happen in a specific time zone (meetings, contracts, subscription billing, etc.) In those cases you have to record the local time and timezone name (plus disambiguation for DST switching) and figure out how to handle changes of the timezone definition (e.g. by updating denormalized UTC representations or offsets).
I have encountered some bugs related to time zone with docker in which the displayed time in the displayed console log output is different from the local time, which is kind of confusing when reporting issues on github. But if your code isn't done properly and have filters based on local time you need remember to pass the correct time zone configuration when shipping with docker.
When reading blog posts / social media / messaging apps you need to be aware of the "responded at xxxx time" usually is local relative to the poster, which may cause confusing anachronism, when trying to know if the message is before or after an event. (If you get the time via an API, it usually is UTC, but when you are scrapping you need to be careful).
Also with DST you always need to check that your alarm clocks have the correct date so they change correctly.
Semi-related, on a dual booting linux/windows, my windows machine after reboot is usually still in the past at the time when it was last shut off, which prevents logging into Fusion360 (without signaling why the server refuse the connection), until my windows get it's time at the right time by synchronizing with the ntp time server.
What if I live on the East Coast and fly to California and back and what to see how my body reacted to the trip? Well some times will happen twice (and get overwritten), some will never happen, and who knows when the change happened in the middle (probably when you happened to sync your Fitbit, sometime after your phone changed timezones).
(1) To complain about
So:
1)We decided to store all timestamps as UTC. A bunch of analysis we were doing was temporal so it was really important to be able to compare timestamps coming from different source systems (with different native timestamps) accurately.
2)Some events were things occurring on specific dates rather than at a particular time. No problem, dates don't have a timezone. We can't possibly be bitten by a timezone bug on dates.
3)One of our serialization formats didn't have a date type so we decided to store dates as times with a timestamp of midnight (This is ancient Pig-era hadoop so you're probably talking an earlyish avro format but I can't remember). You can always just chop the time off to turn them back to a date and nothing can possibly go wrong.
4)One of the event source systems used Europe/London timezone (ie daylight savings in the summer, GMT in the winter) as the system tz rather than UTC. No big deal right, we're doing consistent UTC conversion so this couldn't possibly hurt anything.
Bingo! Some of the date based events appeared one day earlier than they should. Do you see why?
This was an absolute bastard to debug because it was in the middle of a somewhat flakey data pipeline taht used to take >9hrs to run but basically what was happening is during daylight savings months these events were being converted from 00:00 1 Aug BST (say) to 23:00 31 July UTC, so when serialised once you chopped off the time the event would end up one day early. It wasn't all dates because not all days are during daylight savings and events that had an actual time (not a date that had been turned into a time) were of course fine. Also date events that were recorded on a system that used UTC as its system timezone were also fine.
They kind of are, or rather they are time intervals. When people say 25th September they most often mean the interval from 00:00:00 on the 25th until 24:00:00 on the 25th (or 23:59:(59|60) if you don't like one moment in time belonging to two days). This is obviously timezone dependend, at all times some timezones are on a different date than other timezones.