rfc3339 is a profile of iso8601
i.e., every rfc3339 date is also iso8601 date but not in reverse
See the spec: https://tools.ietf.org/html/rfc3339
NOTE: ISO 8601 defines date and time separated by "T".
Applications using this syntax may choose, for the sake of
readability, to specify a full-date and full-time separated by
(say) a space character.
Happily this particular issue is be easily fixed. Let's make a new spec, subsetting both ISO 8601 and RFC 3339.
I hereby introduce... RFC 3339T, which is exactly like RFC 3339, but you have to use a T instead of a space.
EDIT: joshuaissac spotted another difference.
Also I made a repo so we're official: https://github.com/seagreen/rfc-3339T
I made a GitHub repo for the new spec and credited you (https://github.com/seagreen/rfc-3339T). Let me know if you have more suggestions!
An offset of zero, in addition to having the special
representation "Z", can also be stated numerically as
"+00:00", "+0000", or "+00". However, it is not
permitted to state it numerically with a negative sign,
as "−00:00", "−0000", or "−00"."
Thanks for the correction!
This document defines a date and time format for use in Internet protocols that is a profile of the ISO 8601 standard for representation of dates and times using the Gregorian calendar [emphasize is mine]
2021-W07-5
Luxon will happily parse 2021-W07-5T00:00:00Z and tell me that it represents 2021-02-18T16:00:00.000-08:00, so I think it's valid.
I was architecting a true-realtime telemetry pipeline for a AAA videogame studio (later used by a dozen or so franchises for all titles by the publisher). It had a requirement for subsecond client notification of per-user aggregated statistics, including complicated event interactions (such as ensuring the grenade that you threw that killed someone, would only be attributed to you if a prior event hadn't already occured), which resulted in sequentiality guarantees. But with that, we could get rid of windowed event processing (and those inherent latencies), and instead treat them as a true stream of events. Keep in mind we could support up to 400ms one-way latency, so we only had 200ms for processing/routing in the client and in the cloud. I measured client code in μs.
Annnyways. The principal in our org insisted that time not be be transmitted as an epoch. I argued for weeks, but he was insistent... "No magical offsets to arbitrary dates. ISO8601. Do it." He came from and worked most heavily with the web services group. Apparently some point in the past, services had picked the wrong epoch and there was confusion (I still can't understand how that wouldn't be caught right away, there's only a few standard epochs that are quite different in resulting time)? The events in my design were bit-packed using a competing protocol to Protocol Buffers... there's no way I was doubling the payloads just to store a timestamp as a string.
So I read and re-read ISO8601. And it dawned on me: ISO-8601 never specifies the encoding. It specifies the order in which information is conveyed, and if in text, the delimiters to use. One example clue is in the definition of basic format, and the call-out that most of the document leverages plain text to communicate their ideas.
basic format format of a date and time representation or date and time format representation comprising the minimum number of time elements necessary for the accuracy required. The basic format should be avoided in plain text.
I ended up proposing the most terrible hack I've had to live with. It involved bit-packed 64-bit integer that stored: 12 bits for year, 4 bits for month, 5 bits for date, 5 bits for hours, 6 bits for minutes, 6 bits for seconds, and the remaining bits for an integer that stores the sub-second fraction. We lost a few orders of magnitude resolution on that end due to the inefficiencies of the earlier packing, but... it worked. And it was in the right order. And there were no "magic" epochs to dissuade the principal engineer.
Most importantly, it sailed through the review process.
Still makes my skin crawl a decade later. And I secretly suspect you technically have to have a T between date and time to be ISO8601 compliant, it's been a long while since I read the spec. But there you go.
Cross-session reporting (such as calculating DAUs or MAUs) reports on the timestamp from the ingestion service, which is applied at the batch level before a batch of events is placed on the event hub for routing (events are batched and transmitted every 16ms off the client). Typically you can get away with treating everything that happened in the same simulation frame as having happened instantaneously... though we use a monotonically incrementing guaranteed-unique sequence identifier for processing using a high-water-mark (one stream is at-least-once sequentially-guaranteed with retries, while the other stream is lower priority and at-most-once sequentially-guarantied without retries).
No amount of ISO spec will help with client clocks being completely arbitrary. :D The ingestion service clocks are NTP synced to datacenter-hosted atomic clocks, but we still see a few-second skew here and there.
I've even done research into how often they drift backwards in time (but that doesn't really happen, since NTP will just slew the system time by delaying, or adjust the clock massively on startup before the game launches).