How We Use Microservices and Event Sourcing to Instantly Connect Calls
blog.speak.io
blog.speak.io
One thing that has been a big win (which isn't mentioned in the article) is the concept of "duplication of data". Our microservices are currently listening to everything and only taking action on things they care about. As such, they are also storing data about things they care about. This makes it so that no single datastore is a bottle neck in a distributed architecture. This doesn't mean we don't have a centralize DB, but in the heat of operation it doesn't rely on a centralized DB. Using event sourcing makes it very easy to take this distributed data and store it in a consumable way for reporting, etc.
Another thing being it's a much different metric you have to worry about than traditional apps. Instead of worrying about load on a server, your main concern is messages processed per time frame. This can vary depending on what the service is doing, but in general you can simply monitor RabbitMQ's queue processing rate to determine if you need to scale a particular service or not.
It's a bit hard to debug some concurrency problems, but is normally solved by asking: "Who's responsibility is this?" and then adjusting who emits what.
They mention planning out events; we go a bit further and use json schema to describe exactly what is expected in that event. We actually have a dedicated development microservice that simply listens to all events and validates them against the json schema and records/notifies what events (and the source) is not being compliant.
In the beginning of development, our throughput was dreadful, but now that we have all the tools in place, adding a new service or handler of an event in a service is incredibly fast.
One thing that has been a big win (which isn't mentioned in the article) is the concept of "duplication of data". Our microservices are currently listening to everything and only taking action on things they care about. As such, they are also storing data about things they care about.
I couldn't agree more with this statement. I wish I'd gone into this, I was trying to cover a lot here about our new architecture in general.
In the beginning of development, our throughput was dreadful, but now that we have all the tools in place, adding a new service or handler of an event in a service is incredibly fast.
Yeah, we're adding a lot of new functionality and handlers really fast now. The great thing is when we need to split a service out into a new service we simply grab all the handlers for that event and the new service is pretty much already in a proof of concept state!
On top of that, it also makes it easy for front end interfaces to be designed with a mock microservice (or handler) that simply emits it for their testing with a backend developer going through and filling it. As long as the API isn't overly complex for emitting events, almost anyone on the team can do this.
It's more of an approach to higher level architecture than a technique. (think MVC as opposed to Rails). There is no starting MVC tutorial.
Check out https://geteventstore.com/ for a standardized bit of tech with usage docs