Stick to boring architecture for as long as possible | Hacker News Reader

Stick to boring architecture for as long as possible

addyosmani.com

41 pointsmooreds20 comments

I agree with the general message of this article but I want to rant a bit on what exactly is "boring architecture". I haven't quite managed to put this into succinct words, but in my experience building the innards of both backends and frontends for a couple of decades, I found that there's a type of "boring architecture" or "conventional wisdom" that can actually make you build a worse product, slower.

There's nothing odd about having a horizontally scaling (stateless) API server that synchronizes with a database. And then when you need longer running processes, maybe you add in a task queue. But then you want status updates so maybe you set up a pub/sub solution. And then of course your client codebase has to merge all this information into current state. I won't say this is a bad architecture. But if you're a startup with less than 1,000 active users and a handful of engineers splitting their time between working on these systems and your product, I would say to you: how about a single fat server and a transactional database? Maybe I'm old school, but it removes so many modes of failure while increasing performance (except in the rare case that a region goes down) and yet I think people don't tend to look at this option early on if they've been exposed to too much cloudification literature.

And don't get me started on microservices.

* Disclaimer: Every business sooner or later will need to create more distributed systems as they serve more regions, promise more availability, and just generally speaking more users mean more chances for things to go wrong in the infrastructure. But I've found myself several times helping startups revert prematurely distributed architectures to get their current product failing less and running faster, and I would argue it'd be nicer to start stupid simple and expand later.

Premature optimization is the root of all evil

I've been building systems for two decades as well and two lessons:

1) Never act like you're smarter than other developers. This is an industry with a lot of incredibly smart people and with a thousand different ways to solve every problem. There is no right way to do anything. Just the best effort given the cards you are dealt.

2) Anyone who has been involved in re-platforming knows how traumatic, destructive and error-prone it is. Statistically most are a failure. That is why so many developers don't gravitate towards your approach of building a basic architecture and then building something scalable later. They try to do it all in one go. And especially for startups where everyone goes into it assuming they will be building the next Google or Netflix.

Good points, and I agree. I think what bothers me is that a lot of the advice you might read out there is biased towards what you should build if you already had hundreds of employees and millions of users. Sprinkle in a bit of incentives from cloud providers, and you end up with a world where it only seems reasonable to start out with a Kubernetes cluster or multiple distributed communication systems from the get-go. I do think a lot of developers would intuitively start out with a much simpler system, but it almost looks wrong if you do. But it's not wrong, you can't predict the future so if you try to predict and set up your infrastructure for future success, "just in case", you'll still have to re-platform it, as you say.

I should add that some form of re-platforming is pretty much inevitable at some point in the lifetime of a business no matter which approach you take. Because infrastructure is informed by the structure of your organization as much as the needs of your product. If your number of engineers increases by a couple of orders of magnitude, it's likely any infrastructure you set up will need to reshape to account for that.

>Every business sooner or later will need to create more distributed systems

In real, most businesses won't survive till that, even that will survive can go a long way with a high latency service with a single deployment. A well-designed monolith will take to 10s of millions of DAU from across the world. Worst case you'll add 2-300ms of latency to your call, but if you look around most popular and successful of enterprise software have more than that and people are "happy" using them.

> As engineers, we are, by nature, attracted to novel solutions.

I'm not, I'm attracted to proven battle-hardened solutions that have stood the test of time.

I won't dismiss new frameworks and languages and I might try them out in my spare time, but I approach them with skepticism until enough time has passed.

And I'm sure I'm not the only one.

I'm attracted to doing more with less. I find that people are really bad at factoring in all costs, though, and, in particular, the maintenance cost. Battle-hardened solutions are generally easier to maintain and easier to find people who can maintain them. Shiny new frameworks might reduce some immediate costs, but it might not be a good overall solution.

This failure to factor in costs is everywhere. Like people will talk about how fast their car is. Yes, but how much energy is it using? Going twice as fast but using four times as much energy isn't impressive, nor are the enormous environmental and social costs. As a society we are constantly being conditioned to do this, of course. It's basically the whole point of marketing.

Couldn’t agree more. Professionals use battle proven tools. Amateurs reach for whatever is the most shiny tech currently being pushed by cloud providers or open source advocates.

Dan McKinley's original, epic presentation on this subject (now nearly 10 years old?) can be found at https://boringtechnology.club.

It's up there with https://xyproblem.info in terms of URLs I know off by heart.

Depends on what you're trying to achieve.

For Systems of Innovation, well, the whole point is to demonstrate how new technologies may positively impact your application portfolio. The purpose is to achieve organizational experience with the technology. These technologies aren't picked willy-nilly - they're serious contenders for developing applications moving forward.

For Systems of Distinction, the point is the features being developed are unique in the market amongst your competitors and/or provide you a key competitive advantage in your market. Failure represents a high business risk. You'd favor existing, well-known architectures, but you may believe newer technologies could yield much better results. The typical strategy is to take the riskiest part of the implementation and develop it using the newer technology. If that goes well, then proceed with the newer technology. Otherwise stick with the tried-and-true.

For all other projects outside of the above, stick with your current technology stack. The strategy is to get as much delivered with the least amount of implementation risk and having the fewest operational surprises and business disruptions.

Bottom line - technological change needs to be carefully managed in an organization.

Except when doing personal projects for fun! Indeed, this is imho the number one reason (tied with a couple of others e.g. mental health) to do personal coding projects in the first place: it is a safe space to give those shiny new tech toys a go and see how they really perform (sometimes it is: very well! In those cases you know you can then use them for business).

Agree. For some reason many developers thinks that “new” is automatically better than “old”. Which is certifiably crazy.

Old typically means fully bug fixed, with known limits and work arounds, and lots of examples to learn from. And a long track record of solutions using it.

New typically means full of bugs, unknown limits and work arounds, and very few examples to learn from. Plus a high chance of being abandoned and not supported 6 months or a year from now.

Given that, why on earth would you ever choose New over Old? Unless it is for fun personal projects.

To clarify the definition. Boring is in the eyes of the developer writing the solution. What’s boring to the author might not be boring to me. Running a bare metal server with apps and database is not boring, but stressful to me. I find AWS Lambda and RDS, managed thru IaC boring.

This topic is like catnip on here.

The hilarious part is that usually it's about not using Kubernetes, Microservices, NoSQL etc. even though they've all been around for over a decade now. They are boring, stable and well understood by any definition.

I think one of the problems with kubernetes is you have to make a lot of decisions about it. Everything has an alternative. The advantage to this is replatforming is not as risky because you can replatform individual pieces. The downside is that you have to keep making decisions while you are just trying to ship something.

That is one problem, yes. A related problem is it’s entirely too easy to set up a K8s cluster and/or add extras, while having little to no understanding of how the technology functions.

It’s difficult to troubleshoot K8s when you have no idea what cgroups are, or how iptables works, etc. The problem gets worse with managed K8s, because now the control plane is removed from your list of concerns, but it can still cause problems. So now you can still get errors from etcd doing a compaction / defragmentation run, but the logs telling you that are buried, and you probably don’t know what etcd is anyway.

>> As engineers, we are, by nature, attracted to novel solutions

If the context is professional work, then no, I'm afraid. I'm always looking for least suprising, well-understood way. After all, I'm working on a team.

But for personal work, well why not?

I always say something like "Let's make it work so well that it's completely boring. Let's build tomorrow's 'boring'."

The general thought is right, but I've read this a hundred times last year alone and usually in much better presentation.

Boring should not mean underpowered or old.

Using the latest and greatest can equally be boring.

Reply on news.ycombinator.com