Hacker News Reader

Comment by sakisv | Hacker News Reader

Well, you can never be sure that he didn't:

https://www.fastly.com/blog/summary-of-june-8-outage

It's also what was the cause of the Azure Front Doors global outage two weeks ago - https://aka.ms/air/YKYN-BWZ

"A specific sequence of customer configuration changes, performed across two different control plane build versions, resulted in incompatible customer configuration metadata being generated. These customer configuration changes themselves were valid and non-malicious – however they produced metadata that, when deployed to edge site servers, exposed a latent bug in the data plane. This incompatibility triggered a crash during asynchronous processing within the data plane service. This defect escaped detection due to a gap in our pre-production validation, since not all features are validated across different control plane build versions."

This is actually pretty nice and amazing that they publish video format incident retrospectives.

Oh don't you worry. We are very much talking about the global outage as if he was the root cause. Like good colleagues :)

Hmm, wait a minute.. maybe he was the cause! (no, kidding. just upping the pressure as a good peer :)

are we truly good if we don't start a class action suit against this hapless scapegoat?!

Just join the one we've started over in this cubicle!

> May 12, we began a software deployment that introduced a bug that could be triggered by a specific customer configuration under specific circumstances.

I'd love to know more about what those specific circumstances were!

I'm pretty sure I crashed Gmail using something weird in its filters. It was a few years ago. Every time I did something specific (I don't remember what), it would freeze and then display a 502 error for a while.

Damn, imagine being the customer responsible for that, oof

I would be fine if it was my "fault", but I'm sure people in business would find a way to make me suffer.

But on a personal level, this is like ordering something at a restaurant and the cook burning the kitchen because they forgot to take out your pizza out of the oven or something.

I would be telling it to everyone over beers (but not my boss).

I would be tempted to put it on my CV :D

What do you imagine would be the result if you brought down cloudflare with a legitimate config update (ie not specifically crafted to trigger known bugs) while not even working for them? If I were the customer "responsible" for this outage, I'd just be annoyed that their software is apparently so fragile.

Reply on news.ycombinator.com