RFC 3339 vs. ISO 8601
ijmacd.github.io
ijmacd.github.io
https://stackoverflow.com/questions/127803/how-do-i-parse-an...
Parsing ISO 8601 formatted dates doesn't result in naive Python datetimes:
>>> datetime.fromisoformat("2021-10-24T20:17:41+02:00")
datetime.datetime(2021, 10, 24, 20, 17, 41, tzinfo=datetime.timezone(datetime.timedelta(seconds=7200))) >>> from datetime import datetime, timezone
>>> datetime.now(tz=timezone.utc).astimezone()
datetime.datetime(2021, 10, 25, 19, 58, 34, 51668, tzinfo=datetime.timezone(datetime.timedelta(seconds=7200), 'CEST'))
>>> from datetime import datetime
>>> from zoneinfo import ZoneInfo
>>> datetime.now(tz=ZoneInfo("Europe/Berlin"))
datetime.datetime(2021, 10, 25, 19, 58, 34, 51668, tzinfo=zoneinfo.ZoneInfo(key='Europe/Berlin'))datetime.datetime.now() is by definition local to the system. What motivation is there to not include a timezone with the return value?
from dateutil.parser import isoparse
At a glacial pace. Generally stuff is being added and left to die.
>rarely used features are being kept around instead of booting them out with the next minor release
Or, maybe, being actively maintained? Python desperately lacked corporate interest and paid maintainership, and I'm glad that this starts to change. Entirely volunteer model sounds cool, until you see that people are only interested in adding "cool" stuff and syntactic sugar like walrus operator.
> By mutual agreement of the partners in information interchange, the character [T] may be omitted in applications where there is no risk of confusing a date and time of day representation with others defined in this International Standard
[1] The actual ISO standard is not freely available, but the Library of Congress has free draft standards: https://www.loc.gov/standards/datetime/iso-tc154-wg5_n0038_i... https://www.loc.gov/standards/datetime/iso-tc154-wg5_n0039_i...
Justlikehowinmostcasesepacesmakeiteasiertoreadsomethingbutaren'trequired.
Regardless, ISO 8601 has generally discouraged using spaces within expressions. From ISO 8601:2004, section 3.4.1:
> Unless explicitly allowed by this International Standard the character "space" shall not be used in the representations.
Similarly, from ISO 8601:2019-1, section 3.2.1:
> The character "space" shall not be used in the expressions.
However, through looser interpretation of the standard, "2021-10-24 11:02:03Z" could be allowed if treated as a date representation followed by a space followed by a time representation (note that the formal time representation in ISO 8601:2019-1 is T11:02:03Z, but it allows omission of T for extended format as per 5.3.5).
That is, unless it's in a regulated industry that is required by law to follow the actual standard. Then it disobeys the law most of the time, and "disobey" the market the rest of the time and everything breaks. Also, it leaves the people with a secret law that nobody can know the consequences.
The ISO governance should be changed ASAP. It was never good, and nowadays it's a disaster.
ISO was founded in 1947, and back then very many of its standards were about physical objects – pipe threads, screws, bearings, belts, film, steel, plastic, rubber, textiles, etc – and even today they still have many standards on those kinds of topics. Imagine you are an aerospace engineer in the 1970s, and you need a copy of the new standard ISO 46:1973 (Aircraft — Fuel nozzle grounding plugs and sockets) – if nobody at your work has a copy, you just order it and pay for it with the company's money (expense it, get your boss to sign the purchase order, whatever). Nobody would have a problem with that and you wouldn't think there was anything inappropriate about it either, it would just seem totally normal to anybody in the 1970s.
The problem is that technology and culture have changed and ISO hasn't kept up – especially for software-related standards. We are an industry in which there are lots of independent contractors and self-employed people and small businesses, it is easy to be self-taught, in which multi-billion dollar companies will charge millions for software packages and then include in their code some open source library that someone maintains as an unpaid hobby, in which many employers baulk at paying for things which in prior decades employers would have just paid for without question. The culture has changed from one in which people are used to paying money for physical books in person or via mail order, to one in which they expect to get information from the Internet instantaneously and for free.
And it is probably mostly in software that people complain about this. I could be wrong, but I would guess that for aerospace engineers it is no more an issue today than in the 1970s. Even if ISO were to charge a few hundred dollars for a standards document, it would still be a lot cheaper than most things aerospace engineers deal with, and your employer will pay for it, who cares. At big firms such as Airbus or Boeing, they probably have some kind of subscription so their engineers just get access to all new ISO standards automatically.
Consider also the pace of innovation in software and networking—the internet at large. I think it's both a consequence and a cause of the change in culture you described.
It's just that both regulations were much lighter so much fewer people would be affected, and at that time the world was in an extremely totalitarian mood, divided by two countries that imposed dictatorships wherever they went. But entrepreneurship existed by then just like it does now, and you can bet many people went poorer because they couldn't access the standards.
Also, they were much less accessible. If you couldn't just go to a library to read them because they weren't free, it meant most people could not get them at all. It was not just a matter of getting in a phone and calling the ISO customer center.
In the 1940s, if you wanted to read legislation, there was no downloading it for free off the Internet – either you purchased a printed copy, or found a library that held it – and you may have had to travel some distance to do so, there was no guarantee that your neighbourhood public library did. Most people viewed charging for printed copies of legislation as acceptable, given that it cost money to print them. So, in that sense, legally mandated ISO standards weren't fundamentally different from laws in general. Prior to the 1990s, when the Internet began to change people's expectations, the idea that everyone was entitled to a free personal copy of all legislation was not widely accepted, and few would have labelled that situation as anti-democratic or as "absurd secret law".
> It was not just a matter of getting in a phone and calling the ISO customer center.
In the 1940s, rather than calling the ISO secretariat in Switzerland directly, you would have called your national standards body, who acted as a reseller. Buying standards then was only harder than today in the sense that buying anything back then was harder than today.
Was the idea that everyone is entitled to a copy of all legislation at cost widely accepted? Or did most people believe the the publisher (or the legislature?) should overcharge them and stand to make a profit off it?
I don't know how much "at cost" would be for ISO, but something tells me it wouldn't be an identical 158 CHF for both PDF and paper.
Of course, that doesn't entitle you to a copy, making it available at libraries is already enough.
In the real world, most people follow legislation without ever reading it. People build a rough mental model, based on media sources, government websites, common sense, etc, which divides acts into "probably illegal", "probably legal", and "grey area". If they stick to the "probably legal", they are unlikely to have any legal problems. If they are seriously considering venturing into the "probably illegal" or "grey area" zones, a wise person hires a lawyer and gets some professional legal advice first. The majority of the population has no hope of ever understanding legislation – something people on this website tend to forget, because most people on this website are significantly more intelligent and better educated than the average person is. And even those of us who aren't intimidated by legislation and case law and law textbooks, in the way that the average person is, ought to remember the old adage "A man who is his own lawyer has a fool for a client"–if you don't have the formal training and real-world experience of an actual lawyer, it is easy to make a costly mistake–e.g reading some law literally, and unfortunately you never found the case law which interprets it to mean something quite different from what it literally says. Keep the amateur lawyering as a hobby, maybe some real world situations where the cost of being wrong is low (such as challenging a parking ticket), and rely on a professional for anything actually important.
> Of course, that doesn't entitle you to a copy, making it available at libraries is already enough.
Well, ISO standards are freely available at some libraries. Generally national libraries, major public research libraries, university libraries (some of which are open to general public, others restrict admission to staff/students/etc.) Your local public library probably doesn't have any copies of ISO standards–mine doesn't. But, my local public library doesn't seem to have copies of legislation and case law either – my local public library has very little in the way of serious/professional/academic legal texts, just some very introductory stuff aimed at the general reader.
As far as the official version of federal/national legislation goes – in 1947, in both the US and the UK, that was sold by a government agency overall at cost or even at a loss. ("Overall at cost" meaning, they might sometimes have made a profit on an individual print run, but profits they made on some print runs were balanced by losses on others, so they didn't make a profit overall.)
However, most lawyers preferred to use private editions published by for-profit publishers, which added copyrighted notes providing cross-references to other relevant legislation, important court decisions, etc. So the "raw" version of the legislation was available at-cost but the version most lawyers actually used was not. And that remains true today – most lawyers don't rely on the freely available versions on the web, they use expensive commercial subscription services (Westlaw, LexisNexis, etc) which add lots of very useful privately copyrighted notes–and you really need the information in those notes to properly interpret the law, because if you don't have cross-references to other legislation and case law, you won't know what it really means in practice–and although you could probably reconstruct those notes yourself to some extent (such as by searching free case law databases for references to a certain section of legislation), doing so is laborious and likely to be error-prone (it is easy to miss something important because you didn't use the exactly right search term, etc). So even now, access to the law is not as "free" as many think it is.
> I don't know how much "at cost" would be for ISO, but something tells me it wouldn't be an identical 158 CHF for both PDF and paper.
I think from ISO's viewpoint, it is "at cost", because they are not just including the cost of printing or hosting the download, but also the administrative and editorial costs of producing the underlying standard. ISO is a not-for-profit body and any profit it makes is reinvested into the standard development process. By contrast, when you buy a copy of an Act/Statute from the government print office, you are not paying for the actual running costs of the Congress/Parliament/etc which produced the legislation – that is paid for through taxation – whereas ISO being a private body can't levy taxes.
That's not to say that ISO has to use their current model. They could make all their standards freely available and try to recover the editorial/administrative costs through some other mechanism – charging membership fees to corporations, government grants, etc. However, while those methods are feasible – other standard bodies use them – I think ISO would respond that they would make ISO more dependent upon and more beholden to corporate and government interests than they currently are. I suppose the critical response to that is that ISO already is quite beholden to corporate and government interests, and it is hard to see how it could get any more beholden, but ISO would not agree with that.
Carl Malamud has been fighting this point worldwide with some success for decades. For example here's the state or Oregon agreeing: https://public.resource.org/oregon.gov/index.html
Industries that are more reliant on ISO or other non-free standards are surely worse off because of it.
Yes, and? The draft standard is the pretty much everyone actually uses. For most purposes it's the "real" one.
There are only two possible representations for which "T" is required: hh or hhmm in 'basic' format (i.e. no ":" separator). The former could be confused with a two-digit century, the latter could be confused with a four-digit year. Adding "T" to hh or hhmm solves this.
Any of the following eliminates the "risk of confusion":
- 'extended' format (hh:mm)
- decimal fraction (hhmm,m or hh,h; see 5.3.1.4);
- UTC designator (hhmmZ or hhZ; see 5.3.3), or;
- time shift (hhmm±hhmm, hhmm±hh, hh±hhmm, or hh±hh; see 5.3.4).
It honestly seems like a racket. Businesses collaborate to write a standard that costs hundreds of dollars, and then brand new small businesses can’t afford to assert that they follow them.
That's not to say it wouldn't be better, just that it's not without its own problems.
source: https://www.iso.org/files/live/sites/isoorg/files/store/en/P...
https://catalog.nfpa.org/NFPA-70-National-Electrical-Code-wi...
In democracies it seems taxpayer money can only be had for purposes that please some fraction of the legislators.
Should the Library of Congress nationalize the NPFA?
I got my local library to buy a copy of the National Electrical Code. I can study it... but only at the Reference Desk at the library.
There are some derived texts that are pretty good, that the library will let me have, so long as I renew the checkout, and nobody else wants it.
Hmm.
Though iso isn't really spending its own money from what I can tell, so I don't see where their costs are coming from.
It reminds me of scientific journals that do nothing but charge money for other people’s work. It reeks of rent seeking.
Historically, the purpose was to pay for printing & distribution. That makes no sense today, but that was its historical justification.
2022-05-01T16:00+1000
…but tomorrow’s government abolishes daylight savings time, your birthday event has to change to:
2022-05-01T16:00+0900.
If you’d just said:
2022-05-01T16:00 Asia/Eastern_Siberia
…you’d be fine and everyone will show up at the right time.
https://github.com/kstenerud/concise-encoding/blob/master/ce...
https://github.com/kstenerud/concise-encoding/blob/master/ct...
ISO 8601 and RFC 3339 are only useful for times in the past (which works fine for most internet protocols because they're only interested in recording events).
How does it handle leading zeros in the year, like Long Now likes to do? https://blog.longnow.org/02013/12/31/long-now-years-five-dig...
I'm really digging this system in general. I hope it gets more traction.
https://github.com/kstenerud/concise-encoding/blob/master/ce...
Using a date format for precise time types doesn't make much sense because human readable date formats are inherently tied to orbital mechanics[1] (days) and social constructs (timezones, calendars, laws, etc). Better to just record seconds from an epoch.
I see you mention leap seconds in your specification, but you don't address how to handle time in the future given that leap seconds can't be predicted more than 6mo in advance. This is a key feature of TAI -- no synchronization to earth's orbital mechanics. No leap seconds to keep the sun overhead at noon. This means your time spec will likely be off by seconds when using dates more than 6mo out. It seems like your format is also only useful for recording past dates.
[1] TAI is still tied to earth's frame of reference, of course. At some point when we get more serious about space, timestamps will need to include location and velocity. This is already a problem even on the scale of earth satellites and I don't think anyone's yet created a standard.
- The text format is designed for human reading and input. Nobody is going to understand TAI, or bother to look it up. They're going to enter time like they see on their clock.
- Special time representations like TAI require a bunch of complicated machinery to convert to/from a human-friendly format. It's complicated, requires regular maintenance, and implementations have varying levels of buggy. Keeping it human-friendly in the first place eliminates this whole class of problems.
- Leap seconds are already a thing because of the above two reasons, and they're not going away since we're not going to change humanity's understanding of time to something more precise.
We're perfectly capable of handling future dates beyond 6 months without using TAI (and keeping our clocks in sync well enough to be on time).
It looks like you have a human readable (CTE) and binary format (CBE), and you use human-style formatting for the computer readable binary format. This seems incongruous to say the least, especially when I look at how you're representing other numeric types.
While human readable formats are complex, there are standard functions which generate them from epoch time. It's trivial to do this in a display layer, and strongly recommended architecturally speaking.
Also, leap seconds do go away if you use TAI. That's the entire point of TAI. Leap seconds are a display layer issue, just like time zones. They can be applied when translating an epoch to a localized format.
You cannot have seconds-level granularity with the way you've designed your time format. Sorry, but this is just a fact.
The binary and text encodings both store in Gregorian so that codecs don't have the added responsibility of converting between representations (keeps the implementations simple).
TAI is like lye: useful, but not meant to be handled by humans. And since this format is designed for human usability, human time wins the day.
When I look at human readable logfiles I see a trend towards using epoch seconds for exactly these reasons. It's so easy to convert, it outweighs the anti-usability aspects inherent in formatted strings. I've converted all my own human readable file formats to epoch times for exactly these reasons and my customers seem to strongly prefer it.
I don't understand what is stopping you from converting from tai to human readable and vice versa at the form/input and display/output layers, respectively.
TAI conversions are not equally available or of the same quality on all platforms and technologies, whereas Gregorian time is available pretty much everywhere and is well tested, robust, and easy to use (relatively speaking. Time is never easy).
If it turns out in future that I've made a serious blunder with this policy, I could just add a new type "TAI time" and bump the Concise Encoding spec version (Concise Encoding documents all specify which version of the spec they adhere to). There are still about 30 or so RESERVED type slots in the binary spec so we're not about to run out. We then get the type, just a little late. The alternative is that I make the format needlessly complicated by pre-emptively adding TAI, and risk destroying its adoption. Being right doesn't mean success...
For the birthday party example: it's not a given that I want to reschedule it when the local time zone changes (maybe I want to meet before sundown so we can sit outside? in that case, I would want to adjust the local time, keeping the solar timethe same). Sometimes of course, it is desirable (think of the opening hours of a shop, for example), but that's a separate use case.
Similarly with online events involving international guests: if you move an event to keep the local time unchanged, this change might create conflicts on other people's calendars. It's not necessarily the desired outcome.
Another problem with including a time zone name is that not everybody has the same version of the time zone database. At worst, that means that different people will have different interpretations of which point in time is being described. At best, the result will only be eventually consistent.
Look up the Japanese calendar, if you want some fun: https://en.m.wikipedia.org/wiki/Date_and_time_notation_in_Ja...
My birthday party begins next year on May 1st when both our Siberian clocks say it is 4pm.
Then why support timezones at all? UTC-only is a much simpler way to denote fixed points in time; symbolic timezones are needed when you want to denote a human-friendly local timestamp. Numerical offsets are the worst of both worlds.
Eg: Work at the local library is always 9 to 5 (again “local time”) and you mean precisely one hour after work lets up, regardless of whatever daylight savings or time zone changes occur between now and then.
You need to use either a naive date-time representation coupled with the additional context (“my time”) or use something like lat/long. Or you’re lucky and you live in or near enough to a major metropolis that has its own TZDB entry and it’s virtually impossible for your local time to diverge from that TZ, so you can use that instead (eg you live just outside NYC proper so you use America/New_York as your time zone).
One is a a representation stripped of most ornamentation to be universally agreed upon, the other is a representation with many additional layers of context so it's locally understood.
There will likely always be a trade off between these, as a universal time is useful in all cases for only a small group of people, while a local time quickly loses relevance out of a local context.
China has one official time zone that covers five geographical time zones. Tell me, exactly how does saying "It's 6PM in China" tell me anything useful about the local library? You have to go to a local website and find out what time "in that part of China" the library is open. The timezone was never intended to express that.
If it's really at 2022-05-01T16:00+1000, then it doesn't matter what DST policy changes occur.
1/ my birthday party will start when my watch shows this local time and date
2/ my local time and date are dictated by the legislative body responsible for this geographic area.
But beside that this would make sense when everyone involoved is living in the same timezone, but when coordinating internationally (which tends to be the main focus when the ISO is involved) we really do not want such changes to happen automatically. Everyone in the local timezone is likely to be aware of time changes so they know they have to adapt, but everyone in another timezone might have almost no way of knowing. For them, an event starting at a different (UTC) time is a rescheduling, no matter if it is caused by some timezone adjustment or not. So hiding this behind some tzdb change instead of sending out new dates would most likely mean that they appear at the wrong time.
Also including offsets ensures that the person writing the time knows the right offset. In the last year, I had quite some meetings where the host wanted to be nice and send the time in the local timezone for every participant. Since only timezone names and no offsets were included and the host mostly assumed that their local DST rules apply globally, this resulted in a giant mess. Especially after people informed them about errors, so the times switched back and forth between times assuming DST or not assuming DST. Without seeing the offset, there was no way of knowing which time was actually intended. If a UTC offset is given, most local people can relatively easily verify that it is the expected offset but even more importantly, you can independently convert it to the right zone no matter what the author thought.
It’s ugly whatever you do, unless you use coordinates to specify your time zone (then it’s just hell to decode).
However, you can use CSS to select the fixed-width variant for digits. This is a perfect example of a use case for:
* {
font-variant-numeric: tabular-nums;
}Or is it "infinite": as in, the spec allows for 8-digits years, 10, etc.?
For long timescales (geological deep time) I believe using SI units are more appropriate: https://en.wikipedia.org/wiki/Year#SI_prefix_multipliers
Consider that +002021001 could represent:
a) "Oct 1, 202" as a calendar date (proleptic Gregorian), in 'basic' format (i.e. no "-" separator), 5 digits for year.
b) "Jan 1, 2021" as an ordinal date, 'basic' format (i.e. no "-" separator), 6 digits for year.
Expanded representation (i.e. five digits or more for the year) permits both positive and negative years (and the sign is mandatory for both positive and negative years).
ISO 8601-2:2019 has provisions for numeric extensions. Subclause 4.4.1.2 permits negative four-digit years (years -9999 to -0001).
Note that both of these options require mutual agreement.
All experience thus far suggests "No." We'll be using all of these formats, plus some new ones.
So I gave them a project to do parsing a file with contemporary dates in. Then towards the end I quietly supplied them with a similar file containing dates from the year 10000+.
Whether or not the devs of years past felt it similarly unlikely that their software would live to the year 2000 as we do the year 10000, I don’t know. But it was a fun exercise!
https://html.spec.whatwg.org/multipage/common-microsyntaxes....
I often come across a string and don’t recognize the format and would love to just supply a bunch of examples and make a best guess.
Also, I read through that RFC several times and didn't find where it allows replacing 'Z' with space. Can anyone else find it?
> NOTE: ISO 8601 defines date and time separated by "T". Applications using this syntax may choose, for the sake of readability, to specify a full-date and full-time separated by (say) a space character.
Seems to contradict sections 5.2 and 5.5 in spirit...
https://docs.microsoft.com/en-us/sysinternals/downloads/stre...
filenames that are reserved names on windows (like NUL) or contain characters that are reserved on windows are refused with "Invalid filename", even on platforms and filesystems that don't have the same limitations
this might be desirable for enforcing portability in extensions, but the thing isn't mentioned in the documentation[1] for the api so you had to know about it beforehand (like most extension devs, probably) or find out the hard way when some of your downloads unexpectedly fail (like me)
i was using it on linux in a personal extension i wrote, and i thought it was silly that i had to add extra code to further sanitize and mangle the filenames for an OS i don't even use
[1] https://developer.chrome.com/docs/extensions/reference/downl...
[1] '꞉' (U+A789). On my Windows systems the default typeface renders this identically to a regular colon.
Applications using this syntax may choose, for the sake of
readability, to specify a full-date and full-time separated by
(say) a space character.
I don't know how a standard that was specifically written to resolve the issues resulting from such ambiguities ended up with such a crucial issue.
It says the list isn't exhaustive, but I'm just curious how often others actually embed the timezone rather than just passing around UTC.
That said, I wish the page had actually taken the local time into account. As in your case the times on the page are in my current timezone (CEST) so the "Z", "00:00", "+00:00" is basically just decoration.
~~Even worse, in the intersection we have formats like %h:%m:%s+Z and %h:%m:%s+02:00 that show the same time simultaneously.~~
Edit: it is correct now.
However, for the forward looking/scheduling one one, if you want it to keep the same time across timezone/DST policy changes, you need to the full on timezone designator (i.e. America/New_York), not just the offset or short, ambiguous timezone code (EST).
There's no allowance in ISO 8601 for this, so you need a side channel like another field to communicate this. I've seen other date formats do something like 2021-10-11T12:13:14[America/New_York] to allow for this. Once you do this, it probably makes sense to elide the offset to avoid the risk of consfusing mismatches between it and the timezone.
I have a meeting on June 13th 2026 at 15:00 london time.
This could be 2026-06-13 14:00:00 Z
But if the U.K. drops summer time, or makes it double summer time, the meeting will still be 1500 london time, but the UTC time might change to 1500Z or 1300Z
%F %T %z
(Ie %Y-%m-%d %H:%M:%S %z)
Though adding nanoseconds is also recommended.
You can live test according to Linux man-pages project, release 4.14 at emil.fi/p/strftime (sorry about the css, it seems it's broken a little bit, you can use the "use desktop site" selection to have it behave better).
> which does not match calendar year everywhere
I agree that this feature makes quarter indicator necessary. But it also makes it required that the string is something more complex.
That makes doesn't make it a bad time interchange format for certain use cases, though. Just like everything else with time, one must be aware of caveats. Those include timezones, leap periods, culture, DST, formats, legislation (modern and ancient), etc.
"2021-Q1" can be a completely different and non-overlapping range between regions, and even between companies in the same region.
The numbers 1 to 12 are already used to represent months in formats such as YYYY-MM, so starting this list higher than 12 allows these (certainly rare) representations to coexist. e.g. "2021-12" represents "October, 2021" and "2021-29" represents "Spring, Southern Hemisphere, 2021".
Representations of seasons, quarters, etc. is one of many types of extensions included in Part 2.
These extensions are always used in "YYYY-MM" format (and always include the "-"). For the "MM" field, you use the numbers 21-41 inclusive. The numbers represent, in order, spring, summer, autumn, and winter, in 3 ways: independent of hemisphere (21-24), northern hemisphere specifically (25-28), southern hemisphere specifically (29-32); quarters 1-4 (33-36), quadrimesters 1-3 (37-39), and finally, semestrals 1-2 (40-41).
I wonder if there is any software that actually supports that.
Allowing DIN 5008 simply acknowledges reality. Not acknowledging reality is a good way to make people decide that whatever is telling them they're doing it wrong is just "ivory tower" nonsense and should all be ignored at best, possibly actively opposed.
This may or may not be a problem you need to worry about in your API depending on use and clients. For example, if you're only ever sending times and not accepting them, you can just define that the epoch you return is based on UTC or locally and let clients real with it.
If you don't care about pre-epoch times, then it's a compact and fine representation for the DB, single they all have easy conversion functions to their internal formats