In software architecture you have to recognize when you're adding a rocket stage

Adding extra components to you solution, like Kafka, adds complexity, overhead, maintenance etc. While adding an extra rocket stages reduces load, because you get rid of the extra rocket element as soon as it has been used. It will make the rocket lightweight once it is in final orbit.

The intention is good, but the analogy isn't right, it's actually the opposite :)

I never like analogies in decision making, especially when done by management who don't understand technical details of a solution. "Sure we need Kafka, we can't send a tank ship into space can we?"

You've misunderstood the author's point. It's not about "bad judgement". The extra "rocket stage" is just another metaphor about inevitable "tradeoffs" of simplicity vs complexity.

It's a rewriting of the common adage: Every new solution to an old problem also creates a new set of problems. That's just the tautology part. The question is if you'd rather have the "new problems" instead of the "old problems". That's the cost-benefit analysis part.

E.g. Old car engines carburetors wastes too much gas. New Solution uses more complicated fuel injectors with oxygen sensors and onboard computers to dynamically control the fuel mixture. But now your new solution also created new problems of "oxygen sensors go bad that you have to replace" and "add another supply-chain dependency on computer chips" to assemble completed cars. Ok, got it... we'd still rather have the new problems of replacing oxygen sensors and adding the extra complexity of the ECU computers to gain fuel efficiency.

That's what the author is saying with his last sentence about fully understanding if the extra complexity is worth the cost: >"Maybe it's what you need to get where you want to go, but be clear about the extra weight, operational cost, engineering overheard, mental overhead."

(Chin stroke): I think the key metric for insights on the web isn't whether they're accurate, but whether you can sound smart when repeating it to someone else.

(Not saying I approve of this)

The software equivalent is adding infrastructure components to your project because on one level they help you towards the goal of what you are trying to build, but "no free lunch" applies here too, and there is a downside in terms of overall complexity, debugging times, build times, maintenance cost, etc.

Maybe "no free lunch" and/or "technical debt" already express it, and we don't need a new "weight adds weight" way to describe it, although there is something about the tautological inevitability of it that is a valuable reminder.

Akin's Laws¹ of Spacecraft Design have a few relevant ones:

> 13. Design is based on requirements. There's no justification for designing something one bit "better" than the requirements dictate.

This is not the only one that's relevant. Also 15, 27, 33, 34, 36, 36, 38, 44, and, if you substitute "distributed architecture" for "launch vehicle":

> 39. Any exploration program which "just happens" to include a new launch vehicle is, de facto, a launch vehicle program.

Of course, if a staged design is actually required (and not just because your engineers need to buff their CVs so they can jump out when the project fails under its own weight):

> 41. There's never enough time to do it right, but somehow, there's always enough time to do it over.

¹: https://spacecraft.ssl.umd.edu/akins_laws.html

Sometimes "throwaway work" is still worth doing. Maybe you can spend one month hacking an important feature into the old architecture, to deliver it early while investing 6+ months of work into The Right Way. Deprecating the first version is like dropping a rocket stage.

Adding architectural complexity leads to growth in developer headcount. This requires more support staff, and more communication complexity. The growth follows a superlinear trend, as does the fuel requirement for larger rockets.

Let’s say we have one server running our stuff, and it’s too slow.

One option is, upgrade the CPUs or add more RAM. This seems analogous to “build a bigger engine” in your example. There’s no added complexity (well, mostly, maybe you run into NUMA issues or something).

But some kinds of upgrades are more costly or complex than they appear on the surface.

If we want to add 1 TB of usable storage space, at the physical level that means multiple drives for redundancy, keeping multiple copies of the data for backup, keeping replicated copies of data that are ready to turn on at any moment, and so on. So in the developer’s mind we think “add a single 1 TB drive”, but in reality it might take adding 15-20 drives in numerous servers to accomplish this.

The same thing happens when going from a single physical server to multiple. With a single server you can scale vertically. But horizontal scaling is a different type of complexity, now having to think not only about server redundancy, but redundancy at the network layer, now you need redundant network switches, and it all needs to be configured right to handle any part in the communication path failing (server down, server network card down, server network card port down, bad network cable, network switch down, network switch port down), and all of those require some configuration and protocols, and maybe you’re not familiar with VRRP or LACP on the network switches and also how those interact with VMware load balancing and failover, so now you need a VMware person and a Cisco person and that’s not even considering how you handle split brain scenarios where both servers are still running but can’t talk to each other due to a network partition.

And that’s all a very different kind of upgrade than slapping more RAM sticks in the server and keep rolling.

Thinking about my poor heavyweight rocket that barely makes it to orbit and when I compare them to the sleek elegant designs by Scott Manley that makes it to Mun and back and. Sigh.