Hacker News Reader

Comment by cpncrunch | Hacker News Reader

Indeed, it seems to be a completely unrelated issue. Two major Google outages on the same day for different reasons.

The odds that they would happen simultaneously if they’re completely unrelated seem astronomically small, certainly?

Both are noted as being related to “ongoing migrations,” though AFAICT not related ones. I would bet there’s a human factor connection- e.g., the day before there was a big meeting where a higher-level management gave multiple ops teams go-ahead on their respective plans, resulting in a multiple potentially breaking changes occurring at the same time.

I think it's as simple as a case of the Mondays; you wouldn't roll out a migration like that on a Friday or the weekend, and rolling it out on a Monday gives you the least chances of problem occurring on those dates.

A Monday rollout sounds horrible. Probably the worst day for a rollout.

Why is that?

What about Saturday? Sunday? Friday?

GCP rollouts happen over four days, and don't run on Friday. So, Monday it is!

(there is wiggle room, exception granting, and grandfathering on this policy but it's true for many things)

I think the likelyhoood of two incidents happening in the same period is not astronomically small, and is a variant of the birthday problem. It's a bit counter intuitive but if you have a few incidents during a year, the probability to have two incidents the same week is a lot higher than what you would expect.

https://en.m.wikipedia.org/wiki/Birthday_problem

The birthday problem involves random people with unrelated birthdays. This is like two not-random siblings both calling in sick in the same week. The case of two large Google outrages where the whole service gets conked has much more potential to have a shared or related cause than two birthdays happening at once. You're right that it's not astronomically small, but the odds of them being related seem healthier than the odds of them being unrelated.

Reply on news.ycombinator.com