Hacker News Reader

Comment by taffer | Hacker News Reader

Completely agree with the article. If each piece of code has its own data store, you lose all the advantages of the DBMS: you have to handwrite your joins, your transaction system and get all sorts of problems with cache invalidation, data inconsistencies and n+1 fetching. Encapsulation is important, but there are better ways to do it, such as using the authorization system of your DBMS.

View on HN

Agreed, I think the issues you're touching on are fundamentally difficult problems that don't have a single solution. For example.

* storing complex data and supporting multiple access patterns efficiently (like joins) is hard and involves trade-offs (e.g. normalisation vs denormalisation, benefit vs overhead of indices).

* data consistency is hard, particularly once you are at a scale that requires distributed storage

* durability and disaster recovery is hard

* failover and availability, particularly for a stateful system, is hard

The right solution is completely application dependent. Usually the best approach is to avoid hard problems to the extent your application allows it, e.g. by accepting relaxed consistency, partitioning the data, or limiting the use cases the system supports. Then you can outsource the hard problems that remain to an existing solution (e.g. a transactional database).

Microservices help address some of those problems, but you need to recognize when they don't (e.g. you need strong consistency across multiple services) and adjust your designs accordingly

could you give an example of what data would belong in distinct microservices but need to be joined or need transactional semantics ?

Foreign key constraints seems like an easy example. Not every service needs the power to create and manage user accounts but its nice know that every user id in your datamodel is valid.

The only thing that needs a relational data store are reports. Operational data stores should only be concerned with their own domain and expose behaviors and events. There are exceptions, but if you’re building a complex system, monolithic architectures have a known lifetime while MSA’s tend to mitigate long term coderot.

It’s hard to see this until you’ve built domain driven micro service based systems properly.

The trouble is, if you push state into different services behind APIs, you end up reinventing 50 to 80% of RDBMS anyhow. You need transactions, replication, indexing, integrity, backups etc. etc. You can use an event stream (a substitute for a commit log) but then you need to migrate your event stream, be able to undo events in the stream, etc. It's complexity you really ought to avoid unless you need it for serious business reasons.

Monolithic architectures have tended to last for decades in mature industries. I don't think we have a good handle on the lifetime of microservice architectures yet - typically, when they come into being out of necessity rather than fashion, they're part of a startup that hit a major hockey-stick and needed multiple teams working concurrently without stepping on each other, and microservices serve an organizational purpose rather than an architecture purpose.

I will second what other people say: the database tends to stick around. Code may rot, but the data is reused by the next generation.

> multiple teams working concurrently without stepping on each other, and microservices serve an organizational purpose rather than an architecture purpose.

I so hard agree with this.

Agree you need all of these things and more... but if you could do this above the microservice level, as a piece of K8S or whatever the MSA, then you might have an advance over RDBMS.

The result would be the same features/guaranteed you have with current DBMS, but you get independent deployability, elastic scalability, redundant availability, and other advantages of microservices that are missing with traditional DBMS.

> monolithic architectures have a known lifetime while MSA’s tend to mitigate long term coderot

I'd say it's exactly the other way around: the programming languages and technologies change every five years or so, but the data stays forever, so why couple the data with the technology du jour? I'd rather have my data managed in one place so I can adapt the technology layers that sit on top whenever requirements change, rather than writing joins and transactions in my application code. I'd rather not bother with all the complexity and keep it simple as long as I don't need an Amazon or Netflix scale.

Because business complexity modeled relationally becomes stale over time, reduces agility, and develops massive amounts of code rot/tech debt.

Relational databases are tools to be used where it makes sense. NoSQL tables are tools. Graph databases are tools. Configuration files are tools.

Given your statement, Linux’s entire configuration footprint should be in a relational database.

As any good architect will say in response to a design question, “It depends.”

MSA’s aren’t a panacea either. Event Streams are an interesting development and new patterns are likely to emerge.

But today? I’d focus on reducing complexity with domain-driven design and choosing patterns that enable agility and support separation of concerns. MSA fits very well into that philosophy.

> As any good architect will say in response to a design question, “It depends.”

I absolutely agree with that. However I don't like being strawmanned. I never said linux configuration "should be in a relational database". I said I'd like to have my data in one place to reduce complexity and decoupled from the layers above.

You can for the start use a single database as long as the different services don't use / know about each others tables. Then again you could just use separate db's.

Sure, but as I said, if you split your database, you have to deal with transactions, joins, consistency, over- / underfetching. All this leads to complexity and performace issues you wouldn't have otherwise.

Reliable messaging and eventual consistency eliminates the need for transactions. Performance can be a mitigating factor. Joins are for reporting. In the cloud, performance is just a cost factor.

Eventual consistency makes everything even more complicated and I am by far not the only one who says so [1].

[1]: https://cloud.google.com/blog/products/gcp/why-you-should-pi...

Agreed. One DB to focus resiliency on, but every table or database is owned by a single service and managed using migrations.

You can optimize that away later if necessary.

Given your statement we should all be running GHU Hurd.

You could build a separate ReportService which replicates only the data that's needed for a report in RDBMS.

With a sane database design, the Report service can read user details of all users of interest from the user service and ask the Orders service for the orders of each user (a presumably efficient query), without replicating data.

So you’re guaranteeing O(n^2) runtime?

Relational DBs are really just fantastic at high-performance sorts, merges, and joins. Why do you want to pay the continual ongoing cost of replicating that poorly and inefficiently in your application logic? For each microservice that refers to another in some way?

Reply on news.ycombinator.com