From Code Cowboy to Infrastructure Architect | Hacker News Reader

From Code Cowboy to Infrastructure Architect

circleci.com

56 pointsfortyfivan15 comments

"Go fast and break things" works for companies for Facebook for a lot of reasons. First: $, which they have a lot of. You can literally afford mistakes. Second: Monopoly, which they have a lot of. You can figuratively afford mistakes because your user base won't go anywhere.

If you're not operating at FB scale, quality is often your best asset. People tend to choose things that "just work". Getting things to the "just work" phase takes discipline in developing and testing.

If I could add one thing to the list, it'd be "stop writing clever code". Rather than finding a unique solution that shows your technical prowess, write code that's boring but clear in its intent.

Whatever Zuckerberg's mantra back in the day -- when he didn't have limitless capital or a monopoly on social media -- it doesn't seem that it applies to their software or engineering quality today. Any breakage that seems to happen seem to come from dealing with the user side. Whether these fuckups come from careless expansion without proper consideration, or just the difficulty inherent in making a platform meant to be universal for the world's social communication -- they don't seem to be in the same class of fuckups as poor software design and devops.

What other software is there that has to deal with such diverse user input and diverse expectations of output? Even Apple had problems with just autocorrect for the typing of messages.

“Go fast and break things” means that your software and infrastructure deployment process is so resilient, engineers can, safely and quickly, deploy anything into production and roll back should customers think that the new things are “broken”. “People and processes” usually inhibit the creation of such systems in large-enough companies, not money.

That hasn't been a motto at facebook for years. It's now "Go fast with rock solid infra".

Move fast and break things is a motto for startups that might not be here 6 months from now. Speed is important because surviving as a business is the most important priority.

The point on commit messages is mind boggling to me. I have yet to join a team where I am not the first one that uses non single line commit messages.

It is even crazier when they do write some about their commit, but only in some stupid review tool where I then have to follow a bloody link to get to it from the commit itself. (I'm curious if some jerk thought referral links would help them know when reviews were useful later... Please let that remain the idle joke that it is, btw.)

I'm slowly getting folks to put more in the commit message, but leading by example is taking forever. :(

I'm not sure to what extent this remains common, but certain git clients do or did encourage the user to stick to single-line commit messages.

Honestly, sometimes I'm fine with a single line passed to -m for the command. I don't have a hard rule that all commits must have an email attached to them.

However, unless the single line is "fix typo" or some such, often I'm very interested in why code has been sent up for review. And don't just tell me what you did, but why you did it. If I get one more "increased the retry policy" style messages that doesn't expand on why a downstream component needs us to retry more, I'm going to go insane.

A sister team of mine has gone so far as to templatize this. I'm very sympathetic to the idea, but I don't support it yet. I think I'm coming around, though.

I'm too used to seeing "updates" or "<JIRA_ID>". It's insanity. I wish we could get a common understanding let alone a template.

I subconsciously read that title as: "From Code Cowboy to Architecture Astronaut". Realistically you want to be somewhere in the middle, DRY and all, but ship your product often, first of all.

I'm the writer of this post! I absolutely agree with you.

If you don’t want nightmare operations scenarios, you have to document things. This doesn’t reconcile well with Agile/DevOps/Scrum mentality.

Can you elaborate? As stated, I disagree with this statement, as documentation and clear communication are core to these paradigms.

what is even more important is that your infrastructure should be consistent and self documenting (thanks to Configuration Management this is really easy).

Having a solid infrastructure with failure mode and behaviour that is expected saves a ton of hassle and makes ops "easier".

most of the time, having everything redundant to the level that stuff doesn't break is simpely not possible, so its better to make it break reliably. (a good example of this is hard crashing applications on errors which could manipulate data, instead of trying to continue. This makes your failure behaviour consistent across the enviroment).

https://en.wikipedia.org/wiki/Agile_software_development#Cod...

'In a letter to IEEE Computer, Steven Rakitin expressed cynicism about agile software development, calling it "yet another attempt to undermine the discipline of software engineering" and translating "Working software over comprehensive documentation" as "We want to spend all our time coding. Remember, real programmers don't write documentation."'

Good advice, though it's pretty light on concrete actions, details, or examples.

Reply on news.ycombinator.com