Some views on having your system timezone set to UTC

There are two problems here.

In my experience most people, including most developers but perhaps excluding many HN readers, don't understand what the timezone setting does. The time on all contemporary computers is always Unix Epoch Time, which is close to (but not exactly) UTC. Setting your timezone doesn't actually change anything about how time works on the computer. It just changes how time is printed. That's it.

So the second problem is that our text-based server world, for all its benefits, doesn't cope well with per-user parameters. I mean ... timezone support on Unix is basically buggy. It doesn't matter what TZ I set in my .profile, nevertheless the timezone of logs is in whatever TZ the server is running in. That's pretty stupid. And even simple calculations like "how many seconds elapsed between this log entry and this one" - which is really common - are very difficult, regardless of the TZ setting.

In my own personal perfect world, all logs would be printed in the first column of log files using the epoch time in seconds, and we'd have a command which would convert from the epoch time to the local timezone. So we could do something like:

   tail -f /var/log/messages | grep 'panic' | timestamp

...in which case I would be able to choose the timezone and format of the timestamps to meet the needs of the job i was doing.

So for me, this whole argument over UTC is dumb. It doesn't matter what timezone the system is set to; any zone can be "default". The real problem is that different users have different time formatting requirements, and to date we haven't had tools which deal with that properly.

(even logging all timestamps consistently in ISO format would be a big step up...)

This is a common view I've seen.

It usually starts like this " but we're all in <tz> so we should just use that so no one needs to do any maths."

Then later when operations spread to elsewhere it becomes "but most stuff is still done in <original tz> so we will still use that and everyone in other places will just need to deal with it and use our 'real' tz."

Then when you are operating in 3 or 4 or 5 or more locations it is too late to change anything and you are screwed because now you are in some provincial timezone for 50% of your systems - complete with clock changes - and the rest are a mess of either in UTC or some other TZ local to that server with their own different clock changes to the original TZ. Chaos.

And you can guarantee that none of the log files have TZ in their timestamps because of that original "we're only in one TZ" mindset so why bother saying what TZ it is... From then on every time you look at a log or a database or a web UI or whatever you need to double guess the timezone.

For those that come after you, please please please if you can't use UTC then at the very least always show the timezone in your log files/UIs/etc even of it seems redundant at the time - in the future someone handling an incident or trying to fix a P1 bug on a Friday afternoon will thank you.

The reason I would argue for always using UTC is that local time in a time that has timezones isn't clearly unambiguous. In fall, when you set the clock back an hour, suddenly the time 02-03 in the morning happens twice. So if you have an event at 02:13, it could have happened at either of the two 02:13's that happened during that night, and since you are using server local time, you won't necessarily be able to know which one afterwards.

Whether being able to figure this out afterwards is important depends on your use case, of course.

Even though you don't like to do an addition, I think you must use UTC on servers.

It is for example better to use the UTC timezone when you complain to your cloud provider with a support in another timezone.

It can also help to find poorly developed software that ignores timezones before it's too late.

One issue with having your servers not in UTC will be that you're always wondering whether other software e.g. logging software is using UTC or local time. Also, in your database you should always use UTC if you have daylight saving time or else you might have surprising issues where records are out of order by date during time changes. Also, what happens when you start a collaboration with a university in another country and suddenly have users in another time zone. As someone who's worked on many projects with timezones I've seen many people make arguments for not using UTC. Something always comes along that makes them wish they'd used UTC. If you use UTC for everything then you're always sure it's UTC, and it's easy to make the conversion. Otherwise you're always unsure and the costs of uncertainty can be very high indeed.

I have my doubts, having witnessed the repeated confusions between americans (who mindlessly spew out times in like EST, hey, it's called "standard") and non-americans who then try to convert using google which was not always correct, esp. when they forget about am/pm.

A lot of systems let you choose what timezone you want to use in their UI and you can choose to use your local timezone or UTC. We log everything in UTC but I can still filter or even display logs with my local timezone without any issues or major complications.

Having clocks around for local and UTC and CET and EST/PST and JST means you internalize the offsets really quickly and get a good feel for the ball spinning in space. Make it so that whenever you see your local clock, you see all the clocks.

The best reason for running a system in UTC is that you do want to present dates & times in a user preferred timezone. When you have users in more than one timezone as you do with any internet service, then you substantially reduce your chances of errors if you only convert once and not between two non-UTC timezones.

Edit: The other thing about examining logs is that I don't care what local time it occurred. Once an incident is identified all I care about is the time of the incident which can all be in UTC, my mind is at those hours while reading logs from every source.

The only sensible alternative to UTC for the system time is TAI, but I don't know of an OS that lets you use TAI for everything. Perhaps some specialist OS used for satellites? Being able to take the difference of two numerical values to determine the elapsed time, without having to worry about leap seconds, would be so much simpler.

You could convert to UTC for display purposes, when possible. It's not always possible for times in the future, but that's a concept people are already used to: you can't reliably convert a future time stamp between local time zones, either.

You should always set it to UTC regardless. User interfaces can use local time zone. Even if you're a small mom-n-pop shop, good engineering is long term planning. Let's say you get some 3rd party managed appliance and you work with their techs to troubleshoot an issue, or host something in the cloud but all the stackdriver/cloudtrail is not on your local time. Just use UTC until you have a specific reason not to,even then do it at the user/setting level not a system wide default.

In whatever cron fedora is shipping these days, you can throw TZ= statements in them to run events in other timezones...

pretty useful when you need to track DST on a system which is otherwise all UTC.

I’d argue that the only time you shouldn’t use UTC is when you live in UTC. The point is mainly to ensure that your code and institutional processes handle time zones correctly by ensuring nothing accidentally works correctly if they don’t.

Whatever you do pick, you definitely want to avoid Daylight Savings Time because it’s not just twice a year - it’s a different twice a year depending on which years (laws enacted it, and have moved it multiple times).

What's wrong with the ISO format?

If you are lucky it shows your wall clock time.

If you are not lucky you need to do take into account the shift, which is part of the string.

When we were implementing an ERP solution to a manufacturer based in PST and factories situated in China, we faced period close issues since our servers were in PST timezone and factories in China had to wait for around half a day to have the next period open. Switching to UTC has actually made the different timezones overlap and reduced overtime efforts. Anyone here faced similar issues?

And then you got customers in another timezone and suddenly you're in a big mess.

> In fact I think we are a great example of a worst case for using UTC as the system timezone.

I don't understand this at all. I have run all servers in UTC for as long as I can remember. I've run a few teams with very large server fleets.

Understanding that the time reported by a user needs to be converted is just part of the job. It becomes second nature quickly.

Even if all your users are in your local timezone, you will inevitably need to interact with vendors and systems in other places.

I don't get what calamity the author thinks they are avoiding.

A trick I learned a very long time ago — I set all my servers to Arizona time. They don’t have daylight savings time.

Since I’m on the west coast, 7.5 months of the year we’re in the same time zone and for the other 4.5 it’s not hard to subtract 1. Crons don’t really need adjustment with the time change in pacific because off peak is still off peak.