Microsoft Exchange stops passing mail due to bug on 1/1/22

Agreed. Microsoft should be doing that already.

Not exactly fuzzing, no.

Most tests are written with a fixed date, or now(). Instead, using now()+delta can help discover future bugs and give you some time to remedy the issue. Often annoying to write tests like that, though, without them breaking for unrelated/timing issues compared to a fixed value.

not especially. Fuzzing would be running the software with an API to test all of the possible dates.

If you expect your software to exist in 6 months, then the input is entirely expected, and therefore not fuzzing.

It's a nice early-warning system for Y2K or Y2038-like problems.

Oh, EMR systems kill patients all the time.

Like, I’m not convinced “turn it off” isn’t an improvement.

From personal experience, this is Epic.

Same for SAP. Seems like it's still 'best practice' to do so:

http://www.sapbasis1solution.com/how-to-prepare-sap-system-f...

    During this time of one hour “Fall Back” on the clock, as is our Basis team best practice (SAP Note #7417, #102088 and others) we intend to gracefully shut down all Production applications prior to 2:00 AM, at which time the “fall back” happens that will revert back clock time by one hour.  We would then wait for clock time to advance past the “new” 2:00 AM Local time.  And start from 2:05 AM EST will bring up applications.

DST has long overstayed its usefulness. I wonder why it's so hard to get rid of?

So I guess your units tests run with the current date and don't test behavior with different dates. That's something which has the potential to hide quite a few date related bugs until it's too late.

What I like to do to avoid this situation is to use property-based testing to test that the code works for all dates in the desired time span. For Python I use hypothesis [1] in combination with freezegun [2] for that. Here is an example of how that could look like:

  from datetime import datetime, timezone as tz
  
  from freezegun import freeze_time
  from hypothesis import given, strategies as st
  
  
  def business_logic():
      # TODO: Refactor before 2038!
      if datetime.now(tz=tz.utc) > datetime(2038, 1, 19, 3, 14, 7, tzinfo=tz.utc):
          raise Exception("This code is not working after UNIX time overflow")
  
      ...
  
  
  @given(dt=st.datetimes(max_value=datetime(9999, 12, 31, 23, 59, 59), timezones=st.just(tz.utc)))
  def test_business_logic(dt):
      with freeze_time(dt):
          business_logic()
  
  
  test_business_logic()

[1]: https://hypothesis.readthedocs.io/en/latest/

[2]: https://github.com/spulec/freezegun

> the CI builds fail on Friday

That seems different from what I understand by "continuous integration".

Doesn't that just hide the bugs? Your tests should pass on any and all date.

The outcome of tests shouldn’t depend on the time of day. Are these integration tests and you are having to deal with external bugs you can’t fix?

Midnight UTC solves most of these problems.

Hmmm. Where are you? Here in the US DST changes happen at 2AM to avoid ambiguities/weirdness with midnight.

Yeah, they converted the DateTime object into a string of the format YYMMddhhmm which for 2022 starts with 2201010000. Then they convert that into a signed long integer or 2,201,010,000

A long integer is 32 bits. Since it's signed, one of those bits are used for the sign and 31 are used to represent the actual number. With 31 bits, you can express numbers from 0 - 2^31 or 2,147,483,648. 2,201,010,000 is larger than 2,147,483,648.

If they used the format YYYYMMddhhmm, then their numbers for every year would have overflowed the field. Because 201,801,010,000, for example, is too bit to fit in the long. Every year past year 99 would be too big.

But, if it's, say 2008 and you realize that putting the thousand and hundreds will always make it overflow, 801,010,000 would not make it overflow. So why not just cut off thousands and hundreds? Because it's not like we'll run into a 1999/2000 problem before the software is replaced.

> date formatting between it

Oh yes. The nightmare of software that reinvent the wheel.

Instead of using ISO 8601 [1] they feel the need to do something else.

I currently suffer with how time was botched in the otherwise great backup program Borg [2]

[1] https://xkcd.com/1179/ - yes, xkcd has an entry for everything

[2] https://www.borgbackup.org/

The comment you replied to is talking about the 400-year-old-cat from the previous comment, not the Exchange bug from the article.

You know, when I started working with computers (many moons ago) and found out about the 2038 problem, I didn't worry about it. After all, that will never affect me, right?

Well, it's 2022 and 2038 doesn't look that far away now. Hopefully I'll be retired by then and won't have to deal with it. But, something tells me that's not going to happen.

I talked to someone who worked in this team. A mere 30 min conversation opened my eyes to the complexity and he wasn’t even scratching the surface. It’s hard for most people to appreciate the true complexity here. I agree that accountability is needed but even a trillion dollars won’t be enough for bug free code - especially with dates.

I had a weird kind of epiphany when I was watching the fireworks over Sydney last night.

The harbour bridge has lights on it.

For what financial benefit?

The fireworks display itself, I can't understand where the financial benefit is and it costs millions of dollars. ($7 million as far as I'm aware)

---

I think that you could make a financial motivation statement for making good software (it makes it less likely for people to switch, for example), but my broader point is: why does everything have to be financially motivated?

Because, a lot of stuff that we enjoy doesn't seem to be primarily financial in nature.

A lot of people and companies take pride in their work. Saying that people are only in it for a buck is a tired old cliche that is easily disproven through even casual observation.

Considering storage/memory requirements for modern desktop/mobile apps, I wouldn't mind if developers would occasionally think about optimizing them.

Application developers not system engineers. People who work on high performance servers will always look to reduce overhead as much as possible.

There are lot of "numbers" not actually being numbers. E.g. phone "numbers" are not numbers. The same as a house "numbers" are not numbers. Version "numbers" are also often not numbers (e.g. 2.1.230-rc.1).

If the OP bug was really triggered by a string of numerical characters not fitting into a 32bit integer, all the trouble of saving a few bytes on the client system were not worth it.

Re your edit: these days, most developers are web developers where people don't care about these optimizations.

It's a little silly that a test didn't catch this problem, but using a version number somewhere high up in the long makes total sense to me. After all, version numbers can be anything you define them to be, as long as they're unique. You'd like them to be sortable, but they don't even have to be.

But "1000"<"999" in Python.

That would work if padding is always forced (like date). And in this case, it actually is.

2.02 is smaller than 2.12

that is exactly why you would create a transformation to something like a big fat integer with each segment of the version number isolated to separate parts of the bit mask so you can compare it as an integer.

It’s like nothing could have multiple useful forms. Which is of course not true. This is just a dumb bug.

The idea wasn't justifying the choice, but rather trying to imagine what happened. Because they did want a way to sort and pop the highest numeric value, they made a bad choice.

Excel loves replacing phone numbers with 12031041e8 or whatever when importing CSV. So useful.

Unless it is encoded in its design. For US zip codes the first three digits encode a geographic region. But there is no guarantee that 601 is next to 602

pushing the date that it breaks to the version released in 2043?

August 5th.

Similar, but with the important distinction that this problem was recognized months in advance and remedied with some care (or kludge, depending on your preferences), and didn't become a bug in production software.

All computers work in finite representations. All values can overflow. Good engineering is about working within that world and not trying to Quixotically design its absence.

Decide that the null value is one of the invalid digits - e.g., %1111. 0 can remain %0000.

"0\x00"?

F-terminated might work better (given 1-nybble-per-digit BCD), or do a Pascal-style length prefix. I was less fixated on the implementation detail than I am about the basic concept of: arbitrary length numeric strings with numeric semantics and string-like storage and length characteristics.

Honestly that seems like a cop out.

Oh I agree its a bad long term solution, but assuming they had noticed and used the correct type, 2043 is likely well past the lifetime of the product and would have worked fine till then. Not great, but certainly passable.

That is how they are entered by a human, yes. Then they are converted to binary and processed, like everything else. There is no need to add two version numbers together, but as they only ever increasing, its sane and efficient to store, and compare binary.

Everything in software takes data in one form and processes it into another. There are no 'strings', there are no version numbers. There is only binary. They simply did not give enough space to store it, thats all.

DNS is a perfect example, which stores serial numbers internally as an unsigned 32 and has worked for decades and will continue to. But they chose the format of YYYYMMDDnn which will last far longer than a 2 digit year.

And if it's currency, it should be an integer not a float.

To be specific, that representation of date does not implement integer addition. To be fair, neither do most "number" types (uint, int, float, etc).

For uint, you actually have x + y = (x + y) mod MAX_UINT.

For float, sometimes x + 1 = x.

The issue is not that date is not a number, but that people poorly reason about common cases of date arithmetic. Similar stuff comes up for floats and ints but they're just easier to reason about.

Relativity is not modeled in the vast, vast majority of computer systems that track time. So the nitpick is irrelevant.

thank you! this is that I was looking for

And its very doubtful that is all Exchange is doing with it. Likely they have a large list of definitions to compare and track.

you can tell a 4 digit zip code has a leading zero that was removed though. you'll also get faster sql queries if you search for zip codes as int vs string

If you define your version numbers to be numbers, then your version numbers will be numbers. I really don't get the confusion over this.

> 2.9 is an earlier (not less than) version than 2.10.

Not necessarily. Some projects go from 2.9 to 3. I'm not sure what you are trying to prove.

The reference was

> Version numbers the way they’re used here

And none of that applies to them. Yes, there are versioning schemes that work differently, irrelevant to the example here.

Did you read the link that we’re all discussing here? The versioning scheme they use is not any of those formats.

Ignoring the tags (which not everything uses) it's possible to express all of those as a single increasing number by simply putting enough zeros in, which does make them representable as simple numbers, so I think you're kinda splitting hairs. Most version number representations/libraries will likely do something similar to read it anyway even if you keep it a string - split it into separate individual numbers and compare major/minor/patch/etc separately, and by virtue of fitting the individual parts into separate integers you're giving them all a maximum value, which is what the zeros would do.

The problem is they shoved the individual parts into a single 32-bit integer, which just isn't enough space so they had to compromise and make some of the numbers have extremely small ranges.

Google is forcing all Android applications to have both a version name, which is an arbitrary string that's only used for displaying to the user, and a version code, which is a plain integer and is what is used to decide which is the latest version that will be installed.

That should fix a lot of version number problems (prolly not all) and i've been stealing the idea for my own versioning lately.

Character sorting isn't a natural law. The fact that "A" comes before "Z" is historical; it's 'always been done that way'-- at least since the Phoenicians.

https://en.wikipedia.org/wiki/History_of_the_alphabet#Descen...

No they aren't.

10000 can be a valid year according to ISO8601, but sorted as a string it would come before year 2000, which would obviously be incorrect.

Here's from the ISO:

> 3.5 Expansion By mutual agreement of the partners in information interchange, it is permitted to expand the component identifying the calendar year, which is otherwise limited to four digits. This enables reference to dates and times in calendar years outside the range supported by complete representations, i.e. before the start of the year [0000] or after the end of the year [9999].

Which comes first, "2021-W52-6T15:10.4" or "20220101T151115.1-05"? Lexicographic sort between those two ISO-8601 date/time strings will get it wrong.

Did you mistake ISO-8601 for RFC-3339? Even there lexicographic sorting isn't guaranteed, since you can have a pure date or a pure time or a date-time combined, and there's a choice between "T" and " " to separate dates from times in a date-time but it's much more likely to work than with ISO-8601.

I'm curious which languages you learned in CS101 (actual question, not snark)

Every non-number, as you put it, is just a number to a computer. Comparing binary is faster than sorting strings, and it makes zero sense to waste memory simply so a human could potentially read it.

You can certainly sort a string, but I think there are valid reasons to avoid it if possible. What they shouldn't be doing is trying to shove it into a single 32-bit integer - if they parsed it into separate integers for each of the individual parts (like a simple date representation would do) then the issue would go away. Whether it's worth that extra work to avoid the string comparison is debatable, but if they go on to mask out parts of the version number and look at them individually (Ex. Only the year, or only the revision number) than it would probably be worth it to just do all the parsing upfront and avoid any messing with strings after that.

I guess it depends on the language but using a string for this doesn't seem like a great choice except for user display / logging. Given this is a version number you'd want to support comparison operators, and a struct with those operators defined (and a string format method) would make more sense imo.

Depends on the language and function. Some sorting algorithms happily rank ‘9’ > ‘10’.

Fax machines.

Some people treat messaging as having a sync social contract. I’ve had a few people ask me within a min why I’m not responding. I’ve had to set expectations and they thought I was weird.

Or water invented man to carry itself up the hill.

The light belongs to the Sun. Electric lighting is basically cultural appropriation

The drawbacks of accessible computing (e.g. shortened attention span, low-quality media consumption) are vastly outweighed by its potential to allow people social mobility and learn new skills they wouldn't have otherwise.

If I were born in an era before computers and the internet, it's unlikely I would have experienced the best works of classical music and hip hop; explored poetry and the great classical books; and taught myself professional skills and improved academically with access to textbooks and online lecture courses.

I suppose it's possible I could have looked for real-life pockets of communities where these are celebrated, went to a physical library, and asked for help at a community college. But more likely, I would've just copied what my family and peers were doing when I was younger and missed out on so much growth, because of a lack of connection to people passionate about these things that the internet provided.

Around Spectre and other microarchitecture vulnerabilities there was a comment along the lines of "we now forced rocks to think so hard they're getting stress disorders :/"

How do you get to 2038 by increasing by decades only?

Clearly this should stay a string. But I suspect they’re converting to an int to compare which definition is newer.

I think the point is that a 3 digit serial wouldn't be enough to cause overflow, it would need to be 4 digits. But who makes a post like that without using copy/paste to get it exactly right?

I've gone back to the original just to double check. It's very consistent about having only 2 zeros before the last digit, not 3.

Doesn't even have webmail.

You pay very little for these things to be someone else's problem.

You also have to run a datacentre or server room with the on-prem version

HN wisdom is that it's impossible to run a single raspberry pi with less than $2m/year spend on IT staff alone

> Hilariously, the standing advice has usually been for Exchange Online customers to still install and operate a local Exchange server too

Only if you have a large tenant (> 2000 mailboxes) at the time of transition, and even then there’s ways around it. I think if you were getting advice today, it would be to avoid hybrid as much as possible and find any other way to move over because of this stupid requirement.

And RedReader on Android also supports old.reddit links.