Hacker News Reader

Probably not a coincidence

Most places at this scale have code freezes in place well before conferences. The most likely issues are some launch couldn't handle the scale or periodic deployments have been saving them from some sort of long-standing leak bug, and pausing going into Dreamforce meant some service hasn't been restarted in a week. Historically, Salesforce sharded by customer, so that goes against both of these, unless it's in a routing layer.

I'm an ex-Salesforce, and yes, at the time I left, there was a huge change freeze surrounding Dreamforce. Unless a demo of an announced feature was coming in really at the buzzer, change velocity would have been low since a few weeks ago. I worked in a sub-cloud, so I can't even speculate as to the reason for the failure.

Something I wonder about is whether SRE responses were delayed due to having to be emergency-change-approved because Dreamforce was on. I don't recall a global outage ever occurring during a change freeze when I worked there, so /shrug.

Yup. Could be that everyone was rushing to get all their products and demos ready leading up to it.

AWS has a history of this before re:Invents (its annual big conference) in my experience.

They've gotten a little better in recent years.

Reply on news.ycombinator.com