It's led to my personal conclusion that most production issues are caused by people, not errant hardware or systems.
Edited out consideration: traffic in your area might be a good proxy for Uber volume. Traffic late at night is generally low.
From personal experience, having driven for Lyft and Uber I'm two cities (Sf, sd) surge in SF does get acutely high in the morning, more broadly in the evening (work departure times are spread out over a wide range of hours), and acute around 2am. In California, bar closures statewide are 2am at the latest, so that accounts for unified departure times in multiple markets.
It's a well known fact, both for systems and networks.
Not really. DevOps is simply a modernish label for system administration that we have for dozens years already.
Sysadmins have tools to automate their work for a long, long time (cfengine, bcfg2, even Puppet and Chef predates DevOps hype). DevOps didn't bring anything new to the table.
It is also not about hype (or not). I do agree that the name is posterior to the beginning of the practice, but it is the name that we have.
It's still no progress, hacky scripts written by sysadmins or similarly hacky scripts written by programmers, and systematic way of automating tasks was available to sysadmins and was used by them for a long time. DevOps brought nothing new to the table.
We create ourselves so many of the problems we are paid to solve.
Sounds lucrative ;)
I'll bite. :D
It's a bit more nuanced than that IMO, the "deploy often" mantra is only as good as the process around release + deploy. If you half-ass testing and push to production without without a process for verification -- or if, say, your deploy process is half-baked, or your staging environment is worthless -- you can probably expect "interesting" production deploys on a pretty regular basis.
As much as we'd like to pretend we're all good engineers, this happens more often than you'd expect -- even with good engineers: at some point a company transitions from scrappy startup to a shambling beast, and the things that used to work for a scrappy startup (like skimping on testing and dealing with failures in production) are insufficient when you've got more eyes on the product. Further, the engineering culture remains stuck in "scrappy startup" mode long after the shambling has begun.
And all that's ignoring the fact that less frequent deploys with more changes have their own set of problems. We actually got to a point with deploys of a certain distributed system such that we were terrified if we had more than a few days worth of changes queued up. So many things that could go wrong! :)
> We create ourselves so many of the problems we are paid to solve.
This, on the other hand, I completely 100% agree with: if not us, then who? :)
EDIT: minor formatting change
Occasionally we'll have a problem where we cannot deploy to production for several days (normally it's once a day). A massive inventory of ready-to-deploy features builds up. When we do finally deploy, this deluge of features and fixes creates new issues that force a rollback, delaying the deploy even longer...
Man, I'm gonna save so much money.
Most of the other alerts are due to network issues, moving servers due to hardware intervention that change the topology of the network, etc etc.