Raymond Chen explains what the Y2K was like at Microsoft [video]
onmsft.com
onmsft.com
The media - and consequently the public - was generally unaware that it was a non-event because of the massive resources poured into making it so. Instead they wrote it up as a bunch of hype and paranoia over nothing.
IMO it's one of the greatest unsung engineering success stories of our time.
I work with industrial control systems and the oldest code comment that was found about the year 2000 problem was in code from the early 80s. The programmer long retired but that code was still running oil refineries.
There were very little large chemical installations in the pacific so we waited for updates from sites in New Zealand. After those sites rolled into the new year without problem everybody relaxed and we knew that our software installed in middle east oil&gas sites would work ok so people would have petrol and diesel in the new millennium. Then the rollover in the EU and US region were easy after that.
Welcome to the life of any operations engineer, my friend. If we're doing our job right, no one notices that the insanely complex and down-to-the-wire maintenance goes smoothly.
[0] http://www.agsm.edu.au/bobm/teaching/SimSS/Shayne/RepenningS...
Do you remember what that comment was? I mean, was it something like
/* TODO: this won’t work after 1999-12-31, fixme */
/* Using four bytes to handle dates beyond 2000 */Then the next day when I opened ACDSee it said my license had expired. It was a pirated license that previously had shown valid until 2050 or something.
All of a sudden it put the whole thing in perspective, and I realized that those things could happen anywhere causing small bugs, and much bigger ones in systems where dates actually mattered.
(It wasn't the certificate having a date in the future which they will all have done for around a year at that point. It was getting the local time and using that to decide if the certificate was valid.)
Sun Tzu
According to an old story, a lord of ancient China once asked
his physician, a member of a family of healers, which of them
was the most skilled in the art.
The physician, whose reputation was such that his name became
synonymous with medical science in China, replied,
"My eldest brother sees the spirit of sickness and removes it
before it takes shape, so his name does not get out of the house.
"My elder brother cures sickness when it is still extremely minute,
so his name does not get out of the neighborhood.
"As for me, I puncture veins, prescribe potions, and massage skin,
so from time to time my name gets out and is heard among the lords."
- Translator's Introduction, Taoism and The Art of War
The Art of War, Sun Tzu, Thomas ClearyReally? I was only 10 during y2k and not techie at all (yet).
I remember it everywhere...news, school discussions, tshirts, TV, general conversation (similar to zombie apocalypse emergency plans some people talk about).
If anything, I'd say the hype was a little too high. Planes falling out of the sky, nuclear weapons detonating, etc.
Still...I agree. Zero issues. Very impressive.
We knew early on that the Windows dev teams had done their job as Y2K hit Australia and nothing happened. Then Europe and nothing happened.
Even though I "missed out" on the big Y2K celebrations and instead had to celebrate with a bunch of nerd coworkers in a boring office building, it felt good to be a part of something where we all banded together and pulled off a major piece of work.
The Pro customers (non Premier) definitely drove the most support volume and often had the more difficult cases. Premier cases were often "hey what if" cases that we answered quickly or were straight bugs. Believe it or not, bug defects are easy because they're usually obvious and the path forward is also obvious (install checked builds to get more info and/or set up break points and remote debug for crashes).
A lot of work went into fixing the real problems but there also was a lot of fuss about nothing.
EDIT: "for an estimated issue programmers were not taking into account when applying the Gregorian calendar rule to software."
First, what the hell does that even mean? Anyway, no, it was taken into account. What, you think programmers didn't know what would happen when 2000 rolled around? What wasn't taken into account was that the software would still be running ten, twenty years later.
I recall spending significant hours during 98 and 99 on the various systems I was involved with, to check every tick box possible - applying updates where needed and verifying each and every piece of hardware and software.
Number of issues after we'd put in all the hard work: Zero. My reaction to all the people complaining about the lack of issues: Well, doh, _really_? What did you actually expect would happen after so much time and money spent?? Go home.
* Got paid $1000 bonus
* Called up by local reporter asking if any problems - Nope
* Called by several of our vendors/providers in the US just to say "Hello", and to casually ask how things were going.
* One ISP in Australia took themselves offline for the evening
* Rumors were going around Australia that all the power and the phones in NZ had died.
* The one little problem with a provider on the evening got a lot of people closely looking at it.
* There was fog and I didn't get to see the fireworks down town.
One interesting evolution over time is how Windows moved from basically trusting all applications (which they had to do in Windows 3.X anyway, as any badly behaving app could crash the whole system) to treating applications as bad actors.
I was personally also on ready-to-go-to-office stand-by, even not working in Microsoft, but in much, much smaller company. And my team did do a serious work in making the code we were responsible be Y2K proof, we've spent good part of 1998 with that.
In short, we've worked for almost a year to make that "nothing serious happened" result possible, in our own domain of responsibility. The highest management understood the problem and the process was properly planned. That "nothing serious happened" for our code is a success story of the quality of the tests we've did and used to fix the real problems. And they did exist. I'm sure there are other HN readers who can tell the similar stories.
The computer-related risks are a serious subject.
I remember following the news on the New Year's Eve to see what's happening in Japan and Australia. As also "nothing serious" happened there, I was ready to bet that everything will be fine, and that enough other companies also took the subject seriously and acted early enough.
The basic idea was that a bunch of US weapons systems wouldn't work due to Y2K - things like ballistic missile navigation. So the US was frantically trying to get all this patched so that they would have a credible defense after Y2K. The Chinese knew this, and launched on New Years Day...
... and completely missed, because they had used borrowed Russian code for their ballistic missile navigation. The Russians had stolen US ballistic missile navigation code, which had the Y2K issue in it.
Not serious?
https://en.wikipedia.org/wiki/List_of_military_nuclear_accid...
http://arstechnica.com/tech-policy/2013/12/launch-code-for-u...
"Launch code for US nukes was 00000000 for 20 years"
http://foreignpolicy.com/2014/01/21/air-force-swears-our-nuk...
So it was not actually eight zeros but apparently 6 zeroes and the "key under the doormat" (in the safe, but not really something you needed the president to access, the opposite of what was claimed then).
And effectively every computer-related technology has undiscovered bugs.
https://around.com/ariane.html
""The board wishes to point out," they added, with the magnificent blandness of many official accident reports, "that software is an expression of a highly detailed design and does not fail in the same sense as a mechanical system." (...)
(...) really important software has a reliability of 99.9999999 percent. At least, until it doesn't. "
The statistics is against that generous number of nines.
mysql> SELECT UNIX_TIMESTAMP('2037-11-13 10:20:19');
+---------------------------------------+
| UNIX_TIMESTAMP('2037-11-13 10:20:19') |
+---------------------------------------+
| 2141742019 |
+---------------------------------------+
1 row in set (0.00 sec)
mysql> SELECT UNIX_TIMESTAMP('2039-11-13 10:20:19');
+---------------------------------------+
| UNIX_TIMESTAMP('2039-11-13 10:20:19') |
+---------------------------------------+
| 0 |
+---------------------------------------+In the case of Y2K, the problem was often that people defined a date filed as three values all declared as PIC 99 (i.e. a two-digit number). If they migrated to 4 digits, we're fine until Y10K. If they added a window (values less n are 19xx, etc.) or switched to Unix timestamps or a SQL database – there are products which let a COBOL runtime transparently map the indexed file semantics to SQL statements – then it requires more information to say whether it's at risk.
02 ITM-LAST-CHG-DATE.
03 ITM-LC-DATE-CC PIC 99.
03 ITM-LC-DATE-YY PIC 99.
03 ITM-LC-DATE-MM PIC 99.
03 ITM-LC-DATE-DD PIC 99.
+---------------------------------------------+
| TIMESTAMPDIFF(YEAR,CURDATE(), "2050-01-01") |
+---------------------------------------------+
| 33 |
+---------------------------------------------+
The ext4 filesystem in Linux, for example, uses epoch time. I think they have it all fixed now, but there were bugs still out there as recently as this year: https://bugzilla.kernel.org/show_bug.cgi?id=23732
The point is: What do they do? There are plenty that will use... Unix timestamps (or worse). Or use those as the lowest common denominator for interchange. There are more systems at play than the data layer, and all of them will take every opportunity to bite you in the arse.
One of the top results for "convert date to integer". People who have an integer on one hand, a datetime on the other are going to see this code and use it (hey, it works) without understanding the ramifications.
I suppose one can only hope that such code is brittle enough that other problems bring it down before 2038.
postgres> SELECT extract(epoch from to_timestamp('2037-11-13 10:20:19', 'YYYY-MM-DD hh24:mi:ss'));
date_part
------------
2141720419
(1 row)
postgres> SELECT extract(epoch from to_timestamp('2039-11-13 10:20:19', 'YYYY-MM-DD hh24:mi:ss'));
date_part
------------
2204792419
(1 row)So January 1, 2000 was going to be either my 40th birthday, or else my -60th. :)
Or maybe, just maybe it was just a teensy weensy bit overblown so consulting companies could charge really big fees especially to government departments and banks. Easiest way to sell is to scare the st out of people then show them "the solution." I bought an invasion like that once, regret it now...
I personally think this contributed significantly to the dot.com era growth.
Many many companies getting not significant cash inflows in the run up to 2000 which made their balance sheets look great, until the end of the 2000 tax year in 2001 when they didn't look so great any more.
In terms of y2k effort, it seemed to me to be a small amount of code change and a disproportionately high amount of testing. The ramp up in hiring for those changes and certification, combined with the lost opportunity cost, did contribute to a later contraction.
Here's one thing that puts it in perspective though. The amount of code in the world that was Y2K-affected was considerably less than the total amount of code that exists today. And stepping outside the HN startup bubble (where code is relatively young and modernized) and into the types of companies affected by Y2K, there are many efforts that are as bad or worse occurring all the time. Stuff like regulatory changes (SarbOx or various banking reforms), technology uplift (e.g., 32bit --> 64bit, Win32-->.NET, platform ports, etc.) are as bad or worse based on the amount of legacy code it affects.
But what Y2K did is teach a lot of these companies how to solve sweeping, codebase-wide problems.
Quick, do you know, does your computer system ever display the 61st second? And do you think it should?
POSIX:2001 specifies:
"As represented in seconds since the Epoch, each and every day shall be accounted for by exactly 86400 seconds."
And that's typically no problem for normal use. The specialists know that there are different time standards and that for "real" number of seconds one has to use TAI, not UTC.
The problem in Unix world with NTP and the datetime algorithms was that some programmers believed that they have to actually see the leap second on their own computers in the kernel timestamps, up to the kernel intentionally producing discontinuities for kernel times (behavior which never had sense for timestamping purposes but implemented as such anyway). So now we have the configuration variations like this:
https://access.redhat.com/articles/15145
and, to avoid Linux kernel discontinuities:
https://developers.google.com/time/smear
In fact, the smoothing of UTC and using TAI for those who need the "real number of seconds" since point x was known as the reasonable approach long ago:
https://www.cl.cam.ac.uk/~mgk25/time/utc-sls/draft-kuhn-leap...
Now it's clearer why it's complex: too much people locally "assumed" what was not to be assumed and didn't understand the effects of their local decisions and the global context.
Hopefully "smearing" will get standardized and accepted and nobody will have to care, except the specialists who really need TAI. The leap second corrections should be invisible for normal uses, just like nobody cares for correcting the clocks for much bigger differences.