Hacker News Reader

Comment by anotherhue | Hacker News Reader

Micro-anything tends to cause engineering orgs to spend more time drawing boxes and fantasising about what tech would be 'just-perfect' for that particular box than actually considering their users needs.

The killer truth is, the connections between your components start to become more important than the components, and the work to maintain this system is only really justifiable if the components can be worked on in parallel.

But that never happens, and now you spend all your time trying to herd cats. Embrace a monolith, modularise the code, discuss your work with your colleagues so you don't step on each others' toes.

We're here to build systems, not pet-projects/fiefdoms. There is no software architecture that will allow you to work in isolation.

View on HN

Separate services helps a lot with bigger teams. No matter how much you modularize code -- which will gradually fall apart anyway unless you have custom tooling that enforces API boundaries programmatically -- you're still tying people to the same runtime environment, which means you're tying people to the same libraries and underlying framework/operating system. Which means major library upgrades become nonstarters (too much code to test/change), and runtime upgrades are rare and challenging (hello, enterprises still running Java 6/7).

There's a balance and the "every function should be a microservice" approach is kind of insane, but "just use a monolith for everything" falls apart pretty quickly at larger team sizes.

I've seen microservices with inadequate boundaries, much like what you just described with monoliths. This was due to product evolution, or maybe initial requirements were not fully understood when the new architecture was implemented. Only in microservices fixing these boundaries is much more difficult than in a single codebase.

> "just use a monolith for everything" falls apart pretty quickly at larger team sizes.

Absolutely! Don't put everything in a single monolith, develop several mediumliths with lower requirements for coupling code together, but keeping it small enough so that upgrades are bearable.

I'm a fan of libraries, not modules, for hard API boundaries. A library, developed and compiled separately and published with versioning, gives you a lot of the clean separation of services without the networking and system overhead. Dependencies should always point from the app to the library as much as possible, so the library should bring it's own dependencies instead of trying to reuse the app's framework/libraries.

Libraries sort of help but not really. In the end, if the main app is on Python 3.10, all libraries it uses must be capable of running on Python 3.10. And if you want to upgrade the app to 3.11, first you have to upgrade all libraries to support 3.11 (in addition to 3.10, since you'll likely have to do it piecemeal and not a single big-bang).

This is to say nothing of the classic dependency hell[0] where the main app depends on library X and Y, but library X depends on Z==1.0.0 and library Y depends on Z==2.0.0. At which point you have to do some hacky stuff like JAR shading (which has its own problems, see log4shell) or vendoring the dependency under a new namespace.

For a lot of use cases, specifically at larger orgs and where highly optimized performance isn't a big priority, the networking and operational overhead is a small price to pay for decoupling.

[0] https://en.wikipedia.org/wiki/Dependency_hell

Python and any other popular language will try hard to stay backward compatible, so that shouldn't be a problem. I have encountered in the other direction where a library wants to use new language features but the app's runtime doesn't support it yet. That can be considered a downside.

I’ve tried this but did not find it worthwhile in practice. Either you end up running multiple versions of the same library, which sucks terribly unless your library doesn’t have side effects, or you end up doing a huge versioning dance at every change for no benefit

You're right, I didn't give any numbers. IMO for every dozen engineers you are allowed one monolith (but now you call it a service).

> but now you call it a service

What is a problem in itself. On every sane place, most of the "services" aren't providing any on-demand service, instead they are just thematic monoliths doing all of a part of a task.

I'm settling on the opinion that a service-based architecture is always wrong. But people keep calling the good architectures by that name.

Well, you’re almost at the moment of enlightenment.

Yes, all the good architectures are service based. And all service based architectures are wrong. Because all architectures are wrong.

In the sense that there’s always another way you could have architected it that would be better for where you think you want to go next, than the way it is architected.

Learn to accept that an evolving software system is always going to have a suboptimal architecture.

Hum... No.

You are misinterpreting my words here. I'm saying that I now strongly suspect that biasing the way you divide your software on a direction that gives you services will always make your life worse (probably by a lot). There's no philosophical allegory about perfect things not existing.

There are a few things that really want to be services. You'll settle those on services whatever way you decide to architecture your software. But those are very few, and any push into doing it for more than the absolute minimum is harmful.

You say that like it’s a fact but Facebook has a humongous monolith and it works fine with tens of thousands of engineers working on it at the same time

These blanket statements on Hackernews are so frustrating to read. Every one of these articles is bait for these type of comments. The comments fill up with people saying microservices suck, then it fills up with comments saying those comments suck.

Please understand that every environment, company, and project is different.

After years of working with various microservice-based systems, I got the impression that the whole idea of "microservices" means ignoring various nuances of particular products and organizations and just splitting your system in a pinky-sized codebases and services.

You are completely right, every project is different, and that's why the general idea of microservices is bad. Split your services where there are technical or heavy political and organizational reasons to do so, but don't go for splitting as your default solution.

Your code structure should match your org chart is the best advice I've seen for splitting code. Each team in the org maintains their own micro service. They do whatever they want within it, but when going to a different one they go to an interface. It isn't perfect, but it seems to work the best.

Org charts tend to change faster than services can possibly be re-architected in a large scale system that is growing quickly. Honestly you need a critical mass of engineers who can think end-to-end with a product and business-first mentality, they should decide what tradeoffs make the most sense taking into account a broad range of hypothetical future requirements and what makes sense given current domain knowledge. They must do this with real expertise and ownership, not basing it purely on a 3-year roadmap from non-technical PM. Then teams should be organized around the design of these systems (not the other way around). This overall process should be repeated iteratively as the product and business scales.

Until the org chart is reorganized, then what?

Then you reorg the system. Note that this can go either direction - you can also change your system to how you want the org chart to look. Generally it goes one way because engineers rarely have the power to change the org chart, but sometimes we are consulted and can suggest 'improvements'.

You can't fight Conways Law[1]. Or you can fight it I guess, but you will lose.

[1] https://en.wikipedia.org/wiki/Conway%27s_law

> Your code structure should match your org chart is the best advice I've seen for splitting code.

I've seen that so often it now feels like a sarcastic meme to me.

The org structure changes much faster than your code structure does. Good luck getting those two to stay in sync longer than the CTO stays in their role.

There are interfaces besides RPCs

> Please understand that every environment, company, and project is different.

No doubt, but my point is there is a unifying metric that we can use to evaluate the effectiveness of these environments, Percentage of time spent on the environment itself versus on user needs. I think that number is too high for any system that is overly-decomposed.

> modularise the code

You mean…

spend more time drawing boxes and fantasising about what tech would be 'just-perfect' for that particular box?

Won’t that mean the connections between your modules start to become more important than the modules, and the work to maintain this system would only really be justifiable if the modules can be worked on in parallel?

It means the interfaces become important and you can work on the internals in parallel.

Interfaces are hard. If you can avoid doing them that is a good thing (that is the interface is whatever someone who writes both sides of it wants it to be, and it changes as that person feels free). However on large projects you cannot do that, you can try but eventually some interface is used by so much that changes becomes difficult just because of the other code you need to change. (additions are easy, but if you want to change something like the arguments to an existing function that is hard just because of all the users who need to change)

Version your API, and talk to your consumers before building it so you can design an API that meets their needs. Run a pilot with at least two consumers before even announcing it.

Those are good best practices, but they don't solve the problem. If nothing else you end up supporting a V1 API that is now very unlike how you want people to use your system.

Often the pain of supporting this is so high you will just refuse to ask if the API can be improved despite the pain of the bad API.

Just like in a microfrontend or microservice architecture

A function boundary in a compiler is a lot easier to work with than a network boundary for a service in another repo in another language on another kernel. Those false equivalencies are what drive the cost.

Hmm I wouldn't say it never happens we are building a large system (in cybersec space) and there is like a dozen + teams that are delivering their respective frontend functionality as microfrontends. The experience has being fairly reasonable so far. Very likely because each team's app is fairly isolated.

Reply on news.ycombinator.com