Set server timezones to UTC | Hacker News Reader

Set server timezones to UTC

yellerapp.com

82 pointsdanlindley54 comments

Why not use TAI? You get all the same benefits of UTC, but never have to deal with leap seconds.

TAI: https://en.wikipedia.org/wiki/International_Atomic_Time

The second offset may be awkward when comparing logs, network traces etc across systems that you may not control.

Converting between TAI and UTC requires tables that must be periodically updated and distributed to any system that converts between the two. Smooth UTC is a good alternative for most computing applications.

I always assumed you need such a table anyway to know when leap seconds happen...

Yes, but if the machine clock is synchronized to UTC the machine itself doesn't need the table, and you only have to distribute the table to a small number of NTP servers.

I don't think the article points this out, but another strong point in favor of using UTC (or anyways some timezone that (by _definition_) doesn't have DST) is that then you won't have discontinuities in your monitoring timeseries. It's just lovely when you have two datapoints for "cpu usage at 1:30am", similarly it's just lovely when your alerts fire because there's a 1h gap in your timeseries when the clocks jump forward.

The OP sort of touched on this with the point about "some jobs going missing" at the transition points. It's roughly the same issue I guess. Anyway will certainly join in the chant of UTC, UTC, UTC. Not long ago I had to manage some boxes at different colo datacenters where I had some on pacific time, some on central. It's a huge pain figuring out when things actually happened.

I agree with the author on having servers using UTC. The most annoying pain point for me is trying to read a log file and having to perform lots of mental arithmetic to compare it to the local time when I know the local time for an event.

I wrote a simple tool to translate timestamps between timezones on the fly when viewing files, although I've only used it personally, so I am sure there are plenty of bugs. Might be useful to someone else (or roll your own improved version).

https://github.com/billhathaway/catz

Hahaha I loved this! I actually like the short style of this post. Normally I'd argue that consistency is what matters really, but I actually had not thought about daylight savings time... Short and sweet!

My servers in SF UTC-08:00 Some guy at UTC+02:00 set his event to 2pm on 2016-07-20

But I need to use moment.js to display the correct time with UTC. Is there a way to handle multi timezone users without external library?

My understanding is that Javascript Date objects only let you display dates in either the browser timezone or in UTC.

If you want to display dates in another timezone you need to use a library that does that for you or write a function that adds/subtracts hours/minutes based on the offset.

I'd recommend using a library.

I'd personally just keep using Moment. The pain of working with JS Date objects isn't worth saving a few tens of KiB/a few ms of parsing time.

Use "unixtime" ints anywhere you deal with times programmatically and set the TZ to whatever you like...imho the TZ setting is for human benefit, so humans should feel free to change it.

This is of course the right solution in theory, but in practice, tools you don't have much control over use local time, not UTC, when interacting with the user.

Cron is one example. It isn't trivial or intuitive to set up your crontabs in UTC; it should be, because that makes the most sense on a server (especially during DST transitions), but since it isn't, you either have to set your system time to UTC (as the article suggests), hack around it (changing the cron program on all machines you care about), or be aware of the pitfalls (like DST transitions).

From the ISC crontab(5) man page:

The CRON_TZ specifies the time zone specific for the cron table. User type into the chosen table times in the time of the specified time zone. The time into log is taken from local time zone, where is the daemon running.

I use this all the time to set job times relative to a specific timezone while the server time is in UTC. It is trivial and makes setting the times largely intuitive.

I tried this for a while, but it was more confusing.

Logfiles would be written with UTC times, but the time showing on my terminal prompt, or if I ran `date`, was in my local time.

(Possibly more confusing to me, as I live in Western and Northern Europe, with +0000, +0100 and +0200, so it's easy not to notice the time is offset from UTC in a logfile.)

So, is it true that Google for a very long time set their servers to PDT?

Don't they still?

I see headers like this in GMail:

    Received: … Sat, 16 Jul 2016 10:35:05 -0700 (PDT)

which gives me two calculations to get to my own timezone, rather than just one.

The same offset is in headers from Facebook and Microsoft, though Oracle have UTC.

That doesn't say anything about the servers timezones. SMTP is a service on top of those.

It seems unlikely that the SMTP process would run in a timezone different to the server. What benefit would that give?

They would operate a virtual machine-type layer on top of their actual linux machines. In the upper layer they could be doing whatever..

My first VPS is still alive but I made the mistake of setting the timezone to that of my own.

Does anyone have experience with changing timezone of a server?

The only time I have seen time related errors when I extracting a tar file that had a future timestamp.

You need to put appropriate (binary) file to /etc/timezone, or historically, a symlink to a file under /usr/share/zoneinfo/.

"Historically"? You mean symlinks are discouraged/deprecated nowadays?

I only heard about the early-boot issue wish separate /usr partition. Is there anything else?

As for early-boot issue - I'd say it's uncommon to have /usr on separate partition from / nowadays, and those who do this must know what they're doing. Less likely to happen than ad-hoc manual (non-packaged) tzdata update not catching up.

> "Historically"? You mean symlinks are discouraged/deprecated nowadays?

I saw somewhere a comment from a distribution (it was Red Hat, I believe) saying that some programs could have trouble with a symlink. I've never encountered such a program and I could easily live with a symlink, but Debian (my current main distro) went with a file copy, so I didn't bother.

Also, if the user sets a date in the frontend, make sure it gets saved in the right timezone so that the date never changes when viewing it from a browser running in another timezone.

Or, just translate the date from the user's local time to UTC as it is entered and stored, and display it in the local time of whoever is looking at it (or use UTC if the user wants to see it in UTC), so you don't have any confusion.

Sorry, I should have been more specific. I meant to say that if a date is associated with some location (such as on a travel site) then you need to make sure that that the date gets stored for the timezone of that location and that you keep track of what the timezone should be so you can use it when displaying it again.

For example, if you have a flight leaving SFO at July 16, 2016 4:00 PM PDT, you don't want to show it as July 16, 2016 1:00 PM if viewing it from New York.

This thread was about changing time zones though.

If you have a local time and you don't need to change time zones, then of course it is fine to store it as local time.

For events in the future, the mapping between local wall time and UTC is not fixed (consider that Egypt recently cancelled DST on three days' notice), so if the time that matters is local wall time, you should not store your days as a UTC time stamp, you should store it as the local wall time with a zone or location identifier (generally use an Olson TZ if possible).

If you're storing time stamps (i.e. you care about how many seconds have elapsed / will elapse since / until the event), your strategy is the right way to go.

This thread was about changing timezones though.

Of course, if the time you are storing is local, and doesn't need to change time zones, store it as local...

My place uses GMT+4. It fvcks with us every fall.

Saving you the click:

> Setting the timezone to anything other than UTC

All of the arguments in this post supporting UTC could be applied to any timezone without daylight saving.

If your company is entirely located in California or some other Pacific Time location, consider putting your servers on Arizona time. There's no daylight saving, and for most of the year, the server time will match your wristwatch. In the winter, when California is on PST, the servers will only be off by an hour.

I find that much more tolerable than the hassles of UTC servers, where you can't instantly translate between server time and wall time in your head, and server midnight rolls over in the middle of the California day, and thus datestamped logfiles don't match up at all with your wall calendar.

Edit: If you're going to downvote, please post a comment explaining why, so that readers can learn something.

Ugh, daylight savings time and "imperial" measurements are two things I would just order away if I could be granted dictatorial powers for one day.

What if Arizona decides to adopt DST?

It's unlikely to. Residents of a state where summer temperatures get well over 100° regularly don't want summer days to be even longer. The few attempts to do so (for no good reason) have always failed spectacularly, and most Arizona residents are very pleased with it the way it is.

DST had no effect on day length. It has an effect on which actual hours of the day are outside of conventional working hours.

Exactly, and my point still stands. Arizonans want the sun to set sooner after work so they can enjoy a cooler evening and have the hot hours over earlier relative to getting out of work.

What if UTC adopts DST? What if the sun explodes?

UTC won't adopt daylight savings, but Arizona might. This is a foreseeable event which you can mitigate against, unlike the sun exploding.

In the extraordinarily unlikely event that both Arizona and Sonora adopt DST, you can just put an entry in your /usr/share/zoneinfo for Fakeville, US, an imaginary town that honors the time that Arizona and Sonora used to.

You could simply set the TZ variable to a fixed offset:

    $ TZ='<UTC-7>+7' date -R
    Sat, 16 Jul 2016 11:04:52 -0700

Or use Sonora in Mexico:

    $ TZ=:America/Hermosillo date -R
    Sat, 16 Jul 2016 11:05:39 -0700

Another option is Pitcairn Island, which is -0800 all year.

    $ TZ=:Pacific/Pitcairn date -R
    Sat, 16 Jul 2016 10:06:23 -0800

though the standard abbreviation, PST, is probably confusing.

> All of the arguments in this post supporting UTC could be applied to any timezone without daylight saving.

Not this one:

>> I have (in the past), seen servers where the system timezone is set to PST, but the database is set to UTC, causing more crazy bugs, because for 8 hours a day, calling now() in the database has a different date from the application that talks to it.

That isn't an argument against non-UTC servers; it's an argument against using different timezones on your servers and applications.

If your servers and applications are both on Arizona time, it doesn't manifest itself.

It's an argument for a consistent timezone.

If you're exposing an API to (or otherwise sharing data with) third parties, they're unlikely to have standardized on your localish timezone, unless they happen to share it. If you're consuming a third party API (or otherwise consuming data from them), they're unlikely to have standardized on your localish timezone, unless they happen to share it.

If you live in a disconnected bubble, your localish timezone might be consistent enough. You probably don't live in a disconnected bubble. Your localish timezone is not consistent. It is not "Universal". It is not UTC.

> I find that much more tolerable than the hassles of UTC servers, where you can't instantly translate between server time and wall time in your head

Background: we moved all our pan-US servers ( Windows, Unix, AS/400 and MF ) to UTC back in the early 2000s, instead of following the local TZs.

I didn't experience any particular instance when I wanted to convert a server incident to wall-time. Since all events were scheduled in UTC it was a direct look-up between (1) unusual things happening on server and (2) any jobs scheduled at that time. What time it was in the office or in the customer's TZ didn't really matter.

The only time we converted to wall-time was internally within teams to determine when our maintenance shifts would start. With globally-outsourced support teams, the server load charts dictated the UTC time to start the work which was communicated to the appropriate teams who performed local conversion for their shifts. Once they were in the office, everything was executed on UTC.

Most people in this thread don't seem to have experienced how irritating it is to work in UTC. The argument they make is mainly about logging--they basically want to make sure the log line timestamps are in UTC for the sake of avoiding ambiguity. Fine. But when I want to generate a daily report and all the timestamps in the database are in UTC, it's a pain. Measures of time and weight are necessarily conventional so what really matters is consistency.

Couldn't disagree more, I have done extensive work with timezones and UTC where I work and reporting annoyance is a terrible reason for considering something other than UTC. Your front end (be it a browser, cli, or gui) should be responsible for translating dates from the back end to your prefered TZ. Also you indicate that consistency is what matters, I agree which is why I use UTC. Your are completely skipping over DST and how it can mess up other things.

No, consistency is what matters. If you can get away with working with localtime then there's no reason to opt for the faux-universality of using UTC instead.

While it's decent advice, I don't think the reasoning here is particularly helpful.

> DST is different in different timezones, and at some points the overlap will cause a bug when interacting with other systems

That's mostly just an argument for using the same timezone in all of your systems.

> I have (in the past), seen servers where the system timezone is set to PST, but the database is set to UTC

Same deal.

> There are thousands of bugs out there that are caused by timezones

A google search for any other word + bug is going to give you plenty of hits too.

> UTC assuming code interacting with non UTC assuming code can, and has caused silent data corruption

This is an argument for non-naive datetimes. Also is the same regardless of the two timezones involved.

> migrating later than the start of the infrastructure can be really painful, because of your app getting coupled to the timezone it’s in.

That's an argument for not changing timezones, rather than using UTC.

The other couple arguments are similarly moot.

While UTC is good, I think the key is really using consistent date handling. The most important bit is how you store them in the database, and in that regard I'm a fan of fully-non-ambiguous storage. That means either unix timestamps, or timezone-aware datetimes. Ambiguity by assuming a datetime everywhere from a data perspective is where you'll create confusion.

> > DST is different in different timezones, and at some points the overlap will cause a bug when interacting with other systems

> That's mostly just an argument for using the same timezone in all of your systems.

The TZ database is not static. Governments have a habit of monkeying with DST definitions to suit their whims. Several of your systems may have TZ updates come out at different times. Debian/java/erlang/windows have different rollouts for the TZ database that you have to coordinate somehow if you want all your systems to agree on the time/date.

Just use UTC.

I do think you should use UTC everywhere. I just don't think the article makes the best argument for it.

isn't utc the default timezone everywhere?

you're often asked on the setup script, but it mostly defaults to utc if you do an automatic installation and don't specify the timezone.

Reply on news.ycombinator.com