> "all the services needed to talk to each other"
I'm not an expert by any means, but I'm pretty sure that statement indicates a problem.
I'm not an expert by any means, but I'm pretty sure that statement indicates a problem.
And the easiest way to do that is to not build a microservices architecture; instead (and I hope I'm preaching to the choir here) build a monolith (or "a regular application") and only if you have good numbers and actual problems with scaling and the like do you start considering splitting off a section of your application. If you iterate on that long enough, MAYBE you'll end up with a microservices architecture.
What saved us before, was our forest of code could depend on the database to maintain some sanity. And we leaned on it heavily. Hold a transaction open while 10,000 lines of code and a few N+1 queries do their business? Eh, okay, I guess.
Maybe we didn't have the descipine to make microservices work. But IMO our engineering team was pretty good compared to others I've seen. All our "traditional" apps chugged along fine during the same period
Not even the army has perfect discipline, even with hard training. They have cross-checks, piles of processes and move slowly for the most part
(Software development shouldn't aim to be like the army though)
When the services have their own datastores, well now they need to talk to eachother
However a much larger problem was overall bad tooling. Specifically the data storage requirements for an event stream eclipsed our wildest projections. We're talking many terabytes just on our local test nodes.
We tried to remedy this by "compressing" past events into snapshots but the tooling for this doesn't really exist. It was far too common for a few bad events to get into the stream and cause massive chaos. We couldn't find a reasonable solution to rewind and fix past events, and replays took far too long without reliable snapshots.
In the end I was convinced that the whole event driven approach was just a way of building your own "projection" databases on top of a "commit log" which was the event stream.
Keeping record of past events also wasn't nearly as useful as we originally believed. We couldn't think of a single worthwhile use for our past event data that we couldn't just duplicate with an "audit" table and some triggers for data we cared about in a traditional db.
Ironically we ended up tailing the commit log of a traditional db to build our projections. Around that time we all decided it was time to go back to normal RPC between services.
Now I'm seriously considering a somewhat hybrid approach: Collect all of my domain data in one giant normalized operational data store (using a fairly traditional ETL approach for this piece), and then having separate schemas for my services. The service schemas would have denormalized objects that are designed for the functional needs of the service, and would be implemented either as materialized views built off the upstream data store, or possibly with an additional "data pump" approach where activity in the upstream data store would trigger some sort of asynchronous process to copy the data into the service schemas. That way my services would be logically decoupled in the sense that if I wanted I could separate the entire schema for a given service into its own separate database later if needed. But by keeping it all in one database for now, it should make reconciliation and data quality checks easier. Note that I don't have a huge amount of data to worry about (~1-2TB) which could make this feasible.
I'm going against the Martin Fowler grain hard here, but Event Sourcing in practice is largely a failure. It's bad tooling mostly as I mentioned, but please stay away. It's so bad.
Doesn't that imply that each service then has to store any data it receives in these events - potentially leading to a lot of duplication and all of the problems that can come with that (e.g. data stores getting out of sync).