Strftime's alpha-sorted man page vs. well-meaning people
rachelbythebay.com
rachelbythebay.com
I don't think her solution is right, however:
A better fix is to get people away from using format strings altogether.
have one person get it right, [...] then ban all other attempts to use the strings directly
No. Learn the details of your programming language, libraries and frameworks. Even their gotchas. Read blogs, listen to coworkers, learn the idioms.
As a result, you don't store this knowledge in your current corner of some random company. You store it in the ecosystem of the language. It will influence future languages, and slowly the gotchas either disappear or become an near-universal standards.
And I guess that's exactly what this blog post helped to accomplish. Thanks, Rachel ;-)
What do you propose instead?
Of course, they don't cover every possibility - if you want milliseconds, you've got to roll your own :)
[1] https://docs.oracle.com/javase/8/docs/api/java/time/format/D...
I would classify these as storing the knowledge in the ecosystem.
Specifically predefined formats and/or format skeletons. It's absolutely necessary when you want cross-cultural datetimes anyway, because not only are the formats and words different so are potentially the separators/intercalary bits
Javascript's Intl.DatetimeFormat only supports skeletons: you specify which fields you want and in which "width" and it'll get the closest actual datetime format with those fields for the specified locale.
Python's Babel has "named" formats (literally pass "short", "medium", "long" or "full" as the format string to format_date/format_datetime and it'll use the corresponing CLDR format for the locale) and skeletons (format_skeleton & pass in the fields/width you want in LDML specification).
And iso parsing/formatting should be an OOTB primitive of the library, not something you have to hand-roll like a savage.
date parsing is available in strptime. you have to write "%Y-%m-%d %T" (since for some reason only strftime supports %F), but that's not too bad. the major pain is handling locales in a thread-safe manner, which is unnecessarily painful in C.
%c neither lets you specify an overall width nor lets you specify which fields you actually want. Want the shortest possible representation because you don't have much room? Too bad. Want quarters because you actually need them? Fuck you. %c is useless, it's one of the reasons why developers have to resort to hand-rolled formats.
That's exactly what predefined formats provide, what with not being a single specific format. I literally quoted Babel's 4 predefined formats which are only differentiated by their "width".
And skeletons very much let you specify both overall width and field selection (to an extent).
It's not an accusation that the author of strftime didn't have this mindset. Rather, it's to say that we should be more mindful of studies on human behaviors against programming APIs.
Something like, "The ISO 8601 week-based year (see NOTES) with century as a decimal number. Warning: most people want %Y rather than %G; do not use unless you want week-based years. ..."
What if the format codes were just a bit less cryptic? Short words, for each field, optionally qualified to select unusual variants
%{year}-%{month}-%{dayOfMonth} %{hour}-%{min}-%{sec} %{tz:name}
> No. Learn the details of your programming language, libraries and frameworks. Even their gotchas.
Yes, but also, let's reduce gotchas in the design of our programming languages, libraries, and frameworks. It's pathetic that we're still using strftime as our default option in 2018.
but also, let's reduce gotchas in the design
%Y being the correct format to use most of the time is not a "gotcha". If the roles of %G and %Y were reversed I would agree.
Blaming the order of the documentation is grasping at straws.
"This thing that tripped me up for an inexplicable reason wouldn't have done that if the documentation for the thing I wanted to use had appeared first in the document".
IMHO yes, because if they don't pay attention to such things, they're going to make even more subtle mistakes in the future.
Besides the explicit mention to "see NOTES", the fact that most of the format specifiers are mnemonic for the common parts, like H M and S for Hour, Minute, and Second, should be enough to arouse some suspicion when you see G being used for "Year". They weren't assigned arbitrarily --- with the most mnemonic ones being assigned first --- and presumably, people wanted to print years very early in its history, so your next thought should be "then what did they use Y or y for?" If you're really observant, noticing that months and days are lowercase, you may also ask "what's g?" That will naturally lead you to the right choice.
My point is, there's a pattern to these things, and the instinct to ask "why?" and find out when something doesn't feel "right" is very important. More abstractly, "reading requires thinking".
You’re basically saying that programming even basic things should be hard so that people learn to be thorough, but I think that people will have less time to be thorough on the hard parts of programming if 1) even the easy parts require navigating a needlessly complex mass of documentation and 2) programmers have to spend time fixing all of the bugs caused each week by needlessly complex documentation.
In fact, I just asked someone who doesn't know programming at all, after giving her "%m %d %H %M %S" and saying it was a way of formatting a date, to guess how to add a year in front. She guessed... %y. That's actually the two-digit year, which is close enough --- the point is she didn't say %G or some other letter. I'd almost be willing to bet that if you repeated this exercise with many other non-programmers, you'd find the same result.
Then, why would someone be perfectly fine with using %G? To rebut the first sentence of the article, "I have another tale of humans being humans, and getting into trouble when using computer systems", this is not "humans being humans"; it's closer to the "intelligent human losing all semblance of critical thought when put in front of a computer" phenomenon, which certainly deserves further investigation.
Regardless, your test was specifically to confirm your bias that the mnemonic is enough to make a correct decision. There have been plenty of discussions about people blindly following authority. Can you really not imagine someone reading the entry and going "ah, that's weird... But ok"?
But maybe I'm wrong. The M keeps getting bigger, and our reading time keeps getting more fractured. Maybe it's okay to just throw code out without knowing what it does.
Or maybe management/engineering culture need to be clear on the external costs of slapdash development and discourage breaking things fast.
I think at time pressure increases, the grey area between right and wrong grows. Here the hypothetical user did read the fine manual, and they thought they knew what the code did, and when they ran it, the output was what they expected. Really they did everything correctly, except for the fact that they were wrong.
This article is addressing the next level up from RTFM. Given a dev that is reading the manual, how can it be optimised such that they can learn to use the tool correctly? How can project standards be defined so that eg, fiddly date formatting code doesn’t need to be repeated?
I don't think you are wrong. Timing APIs are tricky and complicated. The small print really matters. Experienced engineers will know this, and so the answer in this instance, as annoying as it may be - absolutely RTFM. Also, funky documentation/APIs exist everywhere, unfortunately.
Quoting the article:
> "You're a new coder, and you find yourself tasked with taking a Unix time_t type value and turning it into some kind of human-readable format"
I'd argue that's exactly the kind of mistakes inexperienced coders are prone to make. Hence we have code-reviews.
1. Reference for what exists. This is used by people who don't know what %Ð is a placeholder for. Alphabetic sorting has a big advantage here.
2. Manual for what can be done. This should arguably be grouped differently, e.g. Full dates, years, months, days, etc. since there are often several placeholders for each category that are otherwise pretty far apart at times.
Also the "M"s are generally really, really bad for any practical task. I'd argue that many times they're also bad as references.
They're also not indexed or hyperlinked, there's no tables or any smarter form of formatting. I know that there's also the info format but that's rarely used and it's definitely not default, many distros don't even install the tools for it...
It's a really thankless job to migrate documentation so nobody's going to do it within our lifetimes, so we're stuck with man for the foreseeable future :(
https://github.com/search?q=%22%25G-%25m-%25d%22&type=Code
Plenty of code to go through and fix.
Programming, like life in general, is made out of imperfections. There is a shitload of corner-cases in APIs both ancient and modern that you cannot possibly "fix for all" or, alternatively, learn to avoid. In many cases, it suffices to take that %G after a cursory glance, bump into some weird non-continuous dates later, read the man page again in the heat of a WTF, and change it to %F.
There are applications and cases where dates, and especially the exactness, correctness, and coherency of dates, is of some hugely important factor. Software for banking, accounting, train scheduling, or flying come to mind. In these cases the standard library behaviour can be tested for expected values, or rather, a custom and audited library be written by the team.
But for the majority of programs, things like little glitches in dates, compatibility of encoding conventions in file names and paths, floating point numbers used as decimals etc. will be of little significance. It takes a lot of these to align to unlock one big disaster, and while it is good to strive for perfection there's always a cost to all the little things. If the failures are manageable it is often more practical to just wait for those that will eventually happen.
The "Linux Programmer's Manual" man page from the Linux documentation project has it as:
%G The ISO 8601 week-based year (see NOTES) with century as a deci‐
mal number. The 4-digit year corresponding to the ISO week num‐
ber (see %V). This has the same format and value as %Y, except
that if the ISO week number belongs to the previous or next
year, that year is used instead. (TZ)
But the "week-based year" alone should alert you that something is wrong. Because, like, the year that you know is not week-based! There aren't exactly 52 weeks in a year: 365 isn't divisible by 7, and if it were, then 366 wouldn't be.
A (see NOTES) is also alarming. Why do you have to see some notes about planting the year part of the date into a simple date string?
The glibc manual has it here:https://www.gnu.org/software/libc/manual/html_node/Formattin... It also makes it perfectly clear that this is something weird:
"This has the same format and value as %y, except that if the ISO week number (see %V) belongs to the previous or next year, that year is used instead. This format was first standardized by ISO C99 and by POSIX.1-2001."
Three warning labels here: ISO, C99 and POSIX 2001!
C99 itself gives it like this.
%G is replaced by the week-based year (see below) as a decimal number (e.g., 1997). [tm_year, tm_wday, tm_yday]
If you don't "see below", that is your problem, just like ignoring "(see NOTES)". The "see below" paragraph has this:
%g, %G, and %V give values according to the ISO 8601 week-based year. In this system, weeks begin on a Monday and week 1 of the year is the week that includes January 4th, which is also the week that includes the first Thursday of the year, and is also the first week that contains at least four days in the year. [... more explanation]
Both comments bury the lede. The bit that alludes to but does not actually state "this does not do what you want" is at the end of the description, not the beginning where it might actually warn someone off.
"You didn't read and understand the whole documentation for this complex feature" is not making it easy to do the right thing. It's just fodder for people "in the know" to make fun of normies.
Formatting a year's date into a string doesn't require a complex feature; the "complex feature" part is a warning itself. The date object (struct tm) has a year field that is a simple integer (almost: it's years since 1900). Sticking that into a string in the format "2018" is trivial. The word "weeks" doesn't enter into it and we don't need ISO to figure it out.
Also, you have to be super naive to believe that G was used for "year" instead of the letter Y, and then not even wonder out of curiosity that, if this is how you're supposed to obtain the year, why Y was not available and what the heck Y is being used for. The software field requires people who are curious like this and read documentation critically (as they do code).
Why would someone invent a date formatting printf thing in which %H is hour, but year is %G? Why would they forget to add the year feature, then use Y for something else, and add year support under G in some new standard? Of course, any nonsense lettering is due to historic extension when letters are running out. Why would the basic feature of generating year part of a date string be a historic extension?
> Standards usually sound like a good thing.
The first standard for a programming language or API is often good; then it is usually severely screwed in subsequent revisions. This is massively evident in C, C++, POSIX, Unicode, WWW, IP, USB, Ethernet, ...
Note that make the API is a very distinct activity from document the API, which is what is being criticized here.
A better API would use long identifiers for all weird stuff like %{iso-week-based-year} instead of %G.
> Unix and C culture
I'm confident I could properly understand an above-par engineering document written in plain English, such as what is being criticized here, from any branch of computing.
You can't pin this on Unix and C culture.
Those man pages are documenting this weird ISO "week-based year" about as well as can be. If you read documents in such a manner that you drop random adjectives (such that "week-based year" looks like "year" to your eyes) and ignore anything in parentheses as being optional such as "(see ...)" notices, that's a comprehension problem.
If the descriptions were not sorted alphabetically, that would also be criticized as unfriendly. Users would have to exhaustively scan that section when trying to decipher a format string that someone else wrote.
"This has the same format and value as %Y, except that if the ISO week number belongs to the previous or next year, that year is used instead. (TZ (Calculated from tm_year, tm_yday, and tm_wday.)"
A person whose response is "i don't understand exactly what that means, but it sounds close, so i'm going to use it" should probably not be doing detail work like programming.
When I see a line like that, I check %Y immediately.
Unless I'm tired, in a hurry, or just not paying enough attention.
And it gives away the specifier that you want: %Y.
"Same format as %Y except ..." means "this is some extended/twisted version of another year format, which is assigned to the obvious %Y letter. Hmm, maybe I should check that one out!"
You have to decide what to do with the week that gets broken up when a new year starts, and the ISO standard makes sure everybody uses the same week numbers.
I personally experienced how this mattered when someone in our company was unaware of ISO and decided to invent an other week schema as new standard. First came the people who complained outlook numbered them differently. Then the complaints that all other companies delivered using another system. Then it turns out data transfers from the government didn't match either. Finally one guy pops up and sends the link to ISO in wikipedia.
Our whole 1000 man part of the org was in CC of the original mail, and naturally we received all reply complaints too. After a very noisy few days in the inbox, all of us knew all about ISO weeks and why they matter. Training couldn't be better.
These flags are part of strftime(), so you can generate ISO 8601 week-based dates:
strftime(..., "%G-W%V-%u", ...)
This will generate output along the lines of "2018-W19-7"
> %G The ISO 8601 week-based year (see NOTES) with century as a decimal number. The 4-digit year corresponding to the ISO week number (see %V). This has the same format and value as %Y, except that if the ISO week number belongs to the previous or next year, that year is used instead. (TZ)
I think that's pretty clear, so its actually just a case of not reading / bothering to understand the documentation. I think if you really want to bulletproof a system against incompetency we should probably not start with C.
"I can't explain why I didn't understand this simple, clear piece of documentation. But, aha, if the sections weren't sorted such that it came first before the common thing I was looking for, this wouldn't have happened."
It's just grasping at straws.
There is no way for a documentation writer to predict who will have a momentary brain lapse, reading what passage.
Putting something more toward the back of the document isn't a guarantee that someone won't trip up on it.
I think standard abbreviations are a nice middle ground, at least when a time-zone is needed. Otherwise, just report stuff in UTC.
I presume you’re referring to the fact that CEST is +0200, but +0200 is not necessarily CEST?
https://www.iana.org/time-zones
Accessible copy here: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
For storage of future events, use a proper TZ entry, or suffer like Microsoft did when the US decided to change the date it started summer time and everyone's calendar entries were off by an hour for a couple of weeks.
Turns out there is a period of a few months in I think 1937 where there is a 1 hour difference between them. This messed up birth dates: People get born the day before, at 23:00.
So our software encountered an unholy trinity of bugs,legacy and the oracle SOAP stack. Unless all servers have the same time zone, these people got the birth day mangled when passing data between servers. Which means that superdome infected the time zone for every server created after it.
I'd never knowingly set up a server to use anything but UTC, because the server's timezone shouldn't have any effect on application code and this way, even people in the UK will notice six months of the year.
In this particular case, it sounds like the application was creating data with an implied timezone without any verification that the timezone the _user_ was thinking of matched the timezone of the data, and you may have been better off not associating the point with any timezone -- or indeed with any time, just a date.
I tried for a while to run my local development JVM with an insane default time zone, Japanese character set, etc.. I hoped to shake bugs out of my programs. Unfortunately, almost none of the libraries I have to use survived, so i had to change it back. Sure did find a lot of bugs ;-)
It makes sense for %G to appear with %V, but not with %m and %d.
Nonsensical combinations like this can be diagnosed with a compiler warning if the strftime format is a string literal directly passed to strftime, or found to do so via data flow.
A run-time warning would be nice, but problematic.
Localization is hard, which is why these functions exist.
There is ctime() but it includes a trailing \n which isn’t always what you want.
It's also not useful what with being in a hard-coded US customary format.
%G was added only in C99.
This is just your programmer a) being lazy and not reading the manual (which is short) and b) not writing tests.