Hacker News Reader

Comment by cameronfraser | Hacker News Reader

CQRS and microservices has so much overhead, I'm amazed at how many companies adopt microservices without having anyone know a single thing about distributed systems. I think people underestimate the glue required to make microservices be useful. I had a similar situation at my last company where they spent a year and a half converting their monolith into half ass microservices and it still wasn't working properly. They also stopped releasing new features on their product during this entire conversion which obviously lead to a drop in customers. The mass exodus of employees at the end was something beautiful though.

View on HN

I would argue that main reason for adaptation of microservices is to actually prolong development time (moar dollars) and make devops as convoluted as possible to safeguard data. Or foolishness.

I don't understand the hate for microservices on hn. I have found microservices are a great way to extend monoliths developed with legacy frameworks. There are so many pros with the cons easily avoidable with the right tooling / processes.

Basically it comes down to

Distributed logging:hard

Distributed debugging: hard

Distributed versioning: hard

Distributed transactions: very hard

And on 95% of projects these problems aren't worth solving for the benefits of microservices.

> And on 95% of projects these problems aren't worth solving for the benefits of microservices.

And especially with a team of 4 or 5 developers.

I just started working on a new project about half a year ago, completely greenfield. The backend developer (there is only one...) jumped into microservices directly, deploying on AWS Fargate, trying to split out as many things into containers as possible, because that's the "proper" way of doing it. We still have few users as the project is new, but hey, at least we could scale to 100000s users in a few minutes, instead of having easier logging, debugging and versioning

A lot of stuff in software engineering is just driven by what's popular, not what's needed.

A monolith can handle millions of daily unique users. The database is the hard part.

In the web world, if you have less than 10s of millions of daily users, as long as you design an application that holds no state itself, the architecture is usually more important for scaling your team and the size of your codebase rather than the number of users you can handle.

I've been in discussions about internal applications with few hundreds users and very moderate amounts of data being shuffled and stored and some people, of various backgrounds, are just so convinced that we need to go all in on Kubernetes, Microservices, Istio etc.

And all I can think about is "Hey, you could build this with a small team as a simple monolith and with proper caching you could probably run this on one or two raspberry pi, that is the amount of power you actually need here".

Don't get me wrong I do think they absolutely have their place and in other parts of the company we have much larger software development projects and they are absolutely making great use of Microservice architectures and Kubernetes and is getting a lot out of it. But that is 100+ teams building a product portfolio together.

> And all I can think about is "Hey, you could build this with a small team as a simple monolith and with proper caching you could probably run this on one or two raspberry pi, that is the amount of power you actually need here".

If your product has a global audience who needs to CRUD stuff, caching and a single raspberry pi won't get you very far in the game.

If it's just an intranet stuff with a few hundred users and your front-end isn't very chatty then you're right, you don't need much.

If you don't pile abstractions on top of abstractions on top of kubernetes on top of docker, you'd be surprised how much you can do with a single small-sized instance.

k8s has no performance overhead and can be useful even for simple applications (like for reproducible dev environments and CI). Agreed, otherwise.

I'm not talking about performance overhead, I'm talking about architecture overhead. Kubernetes doesn't have any advantages over not using Kubernetes for simple applications. Reproducible dev environments and CI you can get easily without Kubernetes, without having to add complex solutions for logging, profiling and other introspection tools.

One could argue that you need the reproducible dev environments and CI to be solved BEFORE even start using Kubernetes.

> Kubernetes doesn't have any advantages over not using Kubernetes for simple applications.

So out-of-the-box support for blue/green deployments and fully versioned deployment history with trivial undo/rollbacks are of no advantage to you?

> Reproducible dev environments and CI you can get easily without Kubernetes, without having to add complex solutions for logging, profiling and other introspection tools.

I'd like to hear what you personally believe is a better alternative to kubernetes.

And by the way, Kubernetes does not support not requires distributed tracing tools not "logging, profiling, and other introspection tools". That's somethings entirely different and separate, and something that you only use if for some reason you really want to and make it your point to go out of your way to adopt and use.

In fact, distributed tracing is only a thing not due to kubernetes but due to you operating a distributed system. If you designed a distributed system and get it up and running somewhere else, you still end up with the same challenges and the same requirements.

Apparently HackerNews used to run from a single application server + Cloudflare caching, but later moved away from Cloudflare. Unsure how many web servers it runs on now. [0]

In 2016, Stack Overflow ran on 11 IIS web servers, but they really only needed 1.

[0] https://news.ycombinator.com/item?id=18496344

[1] https://nickcraver.com/blog/2016/02/17/stack-overflow-the-ar...

This.

At least that is what I often think, when I hear people describing micro-services. If there is no data sharing between computations, then the problem is embarrassingly parallel [1] and thus easy to scale. The problem is not the monolith, it is the data sharing, which micro-services only solve if each service owns it’s own data. To be fair people advocating micro-services also often argue that they should own their data, but in quite a few of the instances I’ve heard described, this is not the case.

[1] https://en.m.wikipedia.org/wiki/Embarrassingly_parallel

With microservices, you lose the ability to use database transactions across services/tables. Later on, when people realize this, it's too late.

It's priceless to see their expressions when senior mgmt. and business owners finally find out.

> With microservices, you lose the ability to use database transactions across services/tables. Later on, when people realize this, it's too late.

I don't follow your reasoning. I mean, the ACID vs BASE problem you described is extensively covered in pretty much any microservices 101 course or even MOOC, along other basic microservices tradeoffs such as the distributed tax and ways to mitigate or eliminate these issues like going with bounded contexts. Why do you believe this is a mystery that no one is aware of?

I've seen a couple of projects where the original team didn't think they needed transactions, or underestimated how much harder eventually consistent distributed systems are to reason about than an acid system.

It doesn’t matter if you have transactions or not. Just make sure you execute things in the right order! Same for referential integrity, if you just make sure nothing ever goes wrong, there’s no need for it! /s

Could you elaborate what you mean by transactions?

Sequences of interactions with the DB which are either atomically committed to on success of the sequence, or rolled back on failure, so that the DB is in the same state it had before the interactions began.

Can you explain why you lose that ability? It's not obvious to me.

One of the fundamental laws of microservices is that each service is responsible for storing its own state. Nothing else is allowed access to its backing store.

Given that, once you have more than one service in your architecture, you cannot coordinate transactions across the distinct storage mechanisms - they are distinct databases and are therefore subject to the CAP theorem and other complexities of distributed computing.

Of course, nothing's stopping you from putting all your features into one service to avoid this thorny problem. It's a wise approach for most of us.

When you do that you have a monolithic architecture, not a microservice one.

You can have a monolith and still suffer form these problems and need these technologies, as soon as you introduce a load balancer.

If the application is stateless (all state lives in the DB) you can run as many copies of the monolith as required behind a load balancer.

If your application is stateless following your definition then it's trivial to break the monolith down to microservices that focus on a bounded contexts while sharing a common db. This is a very well known microservices pattern.

https://microservices.io/patterns/data/shared-database.html

Yes, and what does that buy you? Instead of a single deployable unit that owns the database, there are many deployable units, none of which can own the database. That doesn't seem like an improvement.

Martin Fowler lists the benefits of microservices as

Strong module boundaries

Independent deployment

Technology diversity

And with a single database you severely limit the benefits from strong module boundaries and independent deployments. Suddenly you have to synchronize deployments, and anyone can pull or change any data in the database which now must be enforced with discipline which gets you into the same boat as a monolith.

> which gets you into the same boat as a monolith

Only worse because you don't have tools (IDEs, linters, etc) telling you what parts of the code have to be changed.

You can still wind up with distributed state errors. Ex:. User requisitions a billable resource, request hits node A. User immediately terminates billable resource, hits node B. Requisition request has a long latency (and termination request does not). So in the billing system, there is a negative time associated with the resource.

Our initial prototype was a cluster-based approach, and once we had a better estimate of user growth and resources we moved to a monolith for all the reasons you cite.

Why is distributed logging hard? Genuinely curious.

It’s not the log writing that’s hard; it’s the log insight extraction that is. When you compare poring over logs from multiple invocations, even with request identifiers, the tooling is worse than tooling to look over stack traces/core dumps.

A calls B, C, and D. B puts an item on a queue that eventually causes a call to C. Both B and C call D. Which D call is slow/erroring? Is it the AD, the ABD, the ABqCD, or the ACD call?

HN can be a very contrarian community when it comes to popular trends or common practices in the tech industry.

It's just mindless copying. Just because x worked for someone here in this specific situation doesn't mean we should throw out everything else.

No, contrarian is a good description of the problem. We see a lot of posters complaining loudly about problems they never had or tools they never used. Some appear to simply enjoy arguing for the sake of it.

You're assuming malice over incompetence.

Having heard mainly "nothing but guff" justifications for microservices in non-enormous orgs, I think (as usual, and some law I forget dictates) the latter is more likely.

You're thinking of Hanlon's razor.

They're perfect for a startup with lots of funding and little traction. So much busywork to keep the team churning!

Why do people always associate cqrs with microservices?

I have a project where only 3 developers work; we re designed it to be cqrs. I was sceptical at first - I especially don't like some of the boilerplate it creates - but I'm now sold on it by combining it with the mediator pattern, now I can have validation, logging and performance checks on every command and query without repeating code, works like a middleware

And ofc being only 3 devs it would be nuts to have microservices so we are happy with a cqrs monolith

They are adopting microservices for the wrong reasons. It's easy for them to scale up, they don't need to scale out.

Is anyone surprised that a "half assed" implementation of anything doesn't work properly, though?

Reply on news.ycombinator.com