No I'm not. Because the people who need this mentality shift are also people who won't listen anyway and have a negative attitude. And the people who already understand this don't need to see this article.
389 karma · joined October 16, 2012
Talking at the right place at the right time on the right topic is.
How it will actually play out, who knows?
But yes, this is definitely bad for Snowflake, Databricks can position itself as a very strong competitor with this move and moving more towards Iceberg.
But for subjects like Civil, Electrical, sciences like Physics, Biology, I suspect the story is quite different.
Google SRE is overinterpreted. they are a big company with lots of resources and exceptional engineers. It's not reasonable to model smaller companies like that.
For example, The meaning of "S3 was affected" is subject to a lot of interpretation. STS was down, which is a blocker for accessing S3. So, the end result is S3 is effectively down, but technically it is not. How does one convey this in a large org? You run S3, but not STS, it's not technically an S3 fault, but an integration fault across multiple services. If you say S3 is down, you're implying that the storage layer is down. But it's actually not. What's the best answer to make everyone happy here? I cant think of one.
We ran into a very similar issue, but at the database layer in our company literally 2 weeks ago, where connections to our MySQL exploded and completely took down our data tier and caused a multi-hour outage, compounded by retries and thundering herds. Understanding this problem under the stressful scenario is extremely difficult and a harrowing experience. Anticipating this kind of issue is very very tricky.
Naive responses to this include "better testing", "we should be able to do this", "why is there no observability" etc. The problem isn't testing. Complex systems behave in complex ways, and its difficult to model and predict, especially when the inputs to the system aren't entirely under your control. Individual components are easy to understand, but when integrating, things get out of whack. I can't stress how difficult it is to model or even think about these systems, they're very very hard. Combined with this knowledge being distributed among many people, you're dealing with not only distributed systems, but also distributed people, which adds more difficulty in wrapping this around your head.
Outrage is the easy response. Empathy and learning is the valuable one. Hugs to the AWS team, and good learnings for everyone.
History is full of all kinds of examples.