OF MONKEYS AND MICROSERVICES
daemon.co.za
daemon.co.za
Also, how are messages routed to the individual microservices? The author says that each microservice receives a note, and if the note tells it to do something it's capable of doing, it returns a result. So my first instinct is to assume that every message is multicasted to every microservice, and as long as you're careful that no two microservices respond to the same type of message, you'll always get exactly one result. (By "multicast" I just mean "every microservice sees every message and has the opportunity to respond to it," not literal UDP multicasting. This could be accomplished e.g. by sticking each message into a Redis list, which every microservice reads in its entirety. That would also give you a nice deterministic replay log for debugging purposes.)
I'm also guessing that when a microservice generates a result, it returns that result by e.g. stuffing it into a central Redis server, which is being polled by some other part of the architecture which is waiting for the result.
EDIT: Sorry, I meant, does the above design sound reasonable? What are some hidden gotchas? I just came up with it off the top of my head, so it probably isn't great.
Microservice communication will usually work through a message broker such as RabbitMQ or IronMQ - these allow for lightweight routing of messages through subscribed queues, each microservice will listen to an exchange (which has it's own routing rules) which will either round-robin or broadcast to the queues that are attached to it.
In a nutshell, a microservice would set up it's input queues (lashed to exchanges) and it's output queues (pushing to exchanges) and react to incoming messages.
Messages are pushed to specific exchanges by the microservices creating them, they don't care where they go or what happens once they leave, because once they are in the pipe, they are the next service's problem.
You'd want to avoid SPOF such as a Redis instance being polled (the broker deals with that) though you may use it to keep some state information for your various microservices (in case they need to be restarted for example), though ideally, everything should be stateless.
The big problem with microservice architecture is that you potentially receive messages out-of-order, or that events happen across your messaging network in different and unexpected orders, so you need to develop defensively against many different situations if you final output requires a combination of states.
There's actually a whole pattern-based approach to this which is quite well documented in a book called Enterprise Integration Patterns - though it mainly deals with the idea of the Enterprise Service Bus, the patterns should hold true for a microservice architecture.
This is basically where the complex programming will be in these systems, understanding all of the various ways to route messages and where and how to deploy them effectively.
Seneca.js has support for redis, kafka, and so many others.
Martin Fowler's take on them (of "Refactoring" fame) http://martinfowler.com/articles/microservices.html
For a succinct explanation: http://klangism.tumblr.com/post/80087171446/microservices
Hmm, well, I eagerly await the next post.
It seems like what's needed is a concrete, simple tutorial on coding and deploying a microservice infrastructure. That'd demonstrate that this design is practical and, importantly, would reveal any pain points or cornercases that the design is susceptible to.
One pain point is that if a microservice goes down, any messages that were in the process of being handled may be dropped. So there has to be a way to ensure that every message eventually returns exactly one result, and that messages are never accidentally dropped due to microservice crashes.
In so far as things are easier to test in isolation, it'll be due to the more dynamically typed nature of message passing over and above traditional method calls.
"Microservices", or actors or whatever you want to call them, are more justified when you have concurrency inherent in your problem and you want to avoid the trouble threads create.
I am working on a similar blog / articles for a similar implementation - would be interested in sharing war stories - email is in the profile if you are interested
(I was not planning any monkey analogies though :-)
Post wasn't submitted by me, and it's a pity if it was due to the text-transform: uppercase on my headers =)