Developing Service Oriented Architectures
lethain.com
lethain.com
If your call graph goes more than one level deep then doing integration/functional testing becomes much more complicated. You have to bring up all of the services downstream in order to test functionality which crosses that boundary. You also have to worry a lot more about different versions of services talking to each other and how to test/manage that. The flip side is that the services will be much smaller, so leaf nodes in the call graph can reach a level of test coverage higher than a monolithic service.
Debugging and performance testing becomes more complicated because when something is wrong you now have to look at multiple services (upstream and downstream) in order to figure out where the cause of some bug or performance issue is. You also run into the versioning issue from above where you have a new class of bug caused by mismatched versions which either have tweaked interfaces or underlying assumptions that have changed in one but not the other (because the other hasn't been deployed and those assumptions are in shared code). The bright side for debugging and performance is that once you know which service is causing the issue it's way easier to find what inside the service is causing the issue. There's a lot less going on, so it's easier to reason about the state of servers.
Your kind of SOA sounds more like distributed RPC, which indeed is complicated.
However, some kinds of requests are fundamentally about integrating the results of a bunch of different services into a response to send to the user. In that case you somehow need to gather the results of your rpcs/events in one place to integrate them. An example is Google search where the normal results, ads, and various specialized results/knowledge graph data need to be integrated to present to the user.
Another consideration is how much you want to be able to isolate services. If you have a user/auth service as in the article which completely encapsulates the database and other resources needed for data about users then you'll end up with a lot of calls into that service. It's a disadvantage because of all the reasons in my original comment, but it's great from the perspective of being able to isolate failures and build resilient systems
And in the case of auth systems what we typically do is to have a separate app for logins/authentication, then do simple SSO or domain cookie sharing and let each sub system handle the authorization.
My point is that not all SOA has to be as complicated as the article's. But if you go that way, yes, then all your points apply.
We do things very differently to that discussed in the OP (and are heavily influenced by the Krepian school of SOA.) I'd write more, but I'm on a train with flaky internet.
The links section at the bottom alone are a tremendous resource. I've stumbled across most of these over a period of years and if you started with just skimming over this article and the links, you would save yourself many hard lessons.
along with his talk on Programmer Anarchy which is more about the resulting team / working patterns
It's moved playing with much finer-grained service architectures way up my to to list.
Also, HTTP can be a poor choice in security terms for its complexity baggage (cookies, headers, methods, DNS baggage, SSL baggage, etc.). Alternatives such as MQs can be useful to consider, especially at later points in growth, since they can handle complex topologies with ease.
I believe the author could benefit from making a distinction between stateful and stateless in his description of dumb and smart API clients, since state is the main factor resulting from assumptions here.
For me the logical division of services seems to make sense especially when using 'the right tool for the right job'
I may be over optimizing, but I think it will pay off at a later date with more developers and the need to scale each service separately. Maybe someone can point out issues with this thinking (besides those addressed in the blog post).
In a monolithic app interfaces can be more fluid because you can have automated tests and static analysis and compile time checks verifying that a given change works. That means you can prototype and iterate faster while the business requirements may still be churning considerably. To realize the benefits of an SOA you need a much more stable interface and some way to handle validation and correctness of the wire protocol. If you do it right and come up with a stable interface, you gain the benefits of decoupling system administration, scalability, and even development to a great extent. But if you do it wrong you end up a lot more work for an equivalent business logic architecture, and if you don't have any scaling issues than the cost-benefit is likely not to be there.
My rule of thumb about whether an SOA is a good idea to pursue at a given point in time is much more related to the team size than the nature of the app.
(Disclosure: I'm owner of CloudAMQP - RabbitMQ as a Service, www.cloudamqp.com)
Howsabout the one that was addressed in the blog post, namely that SOA is going to slow down your process?
Premature optimization indeed.
A new interface for each service, sure, but if your services only need to be deployed internally, some sort of authentication layer could be overkill.
Nothing wrong with wanting to avoid dependencies. Good programmers try to avoid dependencies and coupling, especially to third party products, all the time.
Service buses like Rabbit are cool n' all but sometimes you don't need the complexity.
Is anyone doing SOA+ESB in a non-Enterprise environment?
Is an ESB actually useful beyond the idea that it is supposed to let "non skilled" people "develop services" (which I'm somewhat cynical about)?
Our philosophy is to use point-to-point when you absolutely need another service's input to complete your task. Otherwise put a message into the bus. See the Jay Kreps blog post I linked in below.
I can see the value in a service locator/registry, but that doesn't seem like a substantial piece of the ESB pitch.
When do you have a fire-and-forget case like you seem to be talking about? I can guess a few secondary concerns like logging, but most of the time if you're calling another service it's because you need a response from that service. At least for the apps I'm used to writing.
i.e. You can have a poorly defined ESB and a big pile of spaghetti with well defined interfaces. Unfortunately I have seen both!
Benefits over monolithic: New languages/techniques can be introduced at any stage, and used for what's best for. Applications can be isolated, and only do one thing, making the system much more reliable. Your web front doesn't have to down because you have a bug in the mailer app.
Benefits over HTTP: Not everything is web facing, workers etc doesn't need a full web stack. Services can be offline, without interrupting other services. You get load balancing and work queuing for free. Performance, 5000msgs/s with disk persistence is no problem for a low end RabbitMQ server. Great inspection capabilities with RabbitMQ's management interface.
Tip: Get away from a "RPC" mindset, think "events" instead. Let other services subscribe to events they should take action to. Decide as much as possible up front, in the application that publishes events, so that information doesn't have to be returned.
Disclosure: I'm the owner of CloudAMQP (www.cloudamqp.com), RabbitMQ as a Service. But have used SOA and RabbitMQ a lot longer, both for small and large projects.
An ESB can be incredibly powerful when you give a small developer team access to it and wait to see what they dig out of the data. We pulled realtime data about sales that no one had access to and it was enlightening for everyone.
When non-programmers use something like tibco, the results are abysmal. Slow overly complicated highly coupled mess that only one person will understand, the creator. When most people say ESB, this is what they mean and it is a tax on the company.
I design and create SOA solutions through services exposed by/through BizTalk and build an ESB solution to orchestrate the services which acts as a substrate for 'composable' applications (again, using BizTalk and apps that use the services are web-based, or whatever client apps the organization wants to use; many apps are integrations between systems like line of business using various approaches that include EDI, etc).
We use it pretty extensively in our SaaS product to achieve loose coupling, yet high cohesion, between our various software components.