I think you bring up a lot of good points, though I'm a bit more optimistic about the trend.
>2) ".... there's a reason people avoid crossing the network, and the microservice approach essentially comes close to crossing the network every single time you cross module boundaries."
We've been trying as an industry to cross the network efficiently for decades. Ultimately it comes down to team structure. Microservices presumes its easier to scale an organization to run/evolve their teams/modules independently from other teams/modules, communicating over a network interface, than "stopping the world" to integrate a monolith or "enforcing a standard across the company" beyond the subset of published language required to make communication work.
Whether that's a reasonable tradeoff generally depends on the performance needs between the modules.
>3) ".... Every single one of them needs to be sufficiently well-designed to operate like an individual web server. So (d)dos-resistance, fairness, anti-slowloris, anti-starvation, autoscaling, resource exhaustion (like maximum number of file descriptors ...) correct sharding, access control (not too tight, not too loose, ... correct caching of security credentials, ...), and load testing, you've thought of all that for that "really small and dumb" service that basically copies information across the payment/non-payment firewall right ? If you haven't ... prepare to be surprised."
I agree. But there are plenty of examples of how to bake that in to each service for reasonable cost - the Netflix OSS services, for example, the emerging platforms like CloudFoundry that give you the resource isolation + access control + scaling + etc. There's no excuse to fail to analyze what's out there and pick an appropriate set to cover these areas.
4) "Last time I consulted at a bank they had systems interoperating using every interchange format from fixed-width fields (old cobol code), 5 different kinds of xml, and of course json and a dozen binary formats. Microservices can't work in such an environment."
Okay I don't really understand this one. Are you suggesting that every micro service should be able to parse all these different formats and semantics? If I needed a micro service that spoke several different languages, I'd probably include some kind of integration library to do such a thing (say Spring integration).
If I wanted to incorporate legacy systems into a new micro service app overall, I'd probably form a micro service that wraps it in a bubble, depending on the circumstances. e.g. http://domainlanguage.com/newsletter/2011-06/
If you look at how Netflix migrated from Oracle / SimpleDB to Cassandra, this latter approach is how they did it (though it was more of a synchronization service than just a bubble, since it was intended to shut off the old system eventually).
5) I agree, the tools for testing and test data replication in the enterprise aren't up to what Netflix pulls off - they rely on the broad facilities of Cassandra and S3 to be able to quickly replicate test data to a cluster.
However, do you have a link for where Netflix says they have 3 teams doing this work? I think this may be open to misinterpretation, because my understanding is that their development teams basically deploy into production - they don't do full integration tests, the canary deploys basically catch boneheaded changes - and there's certainly a lot of A/B testing.
There are testing teams, but they aren't the usual sort: they're building processes to do device testing, "failure injection testing", introducing chaos into the production system. It's just a very different approach to the usual "stop the world and test everything" we see in the enterprise.
"So we simply need an extra image field going through the payment system. Oh-oh. The payment system is 8 microservices, and therefore around 64 interfaces to the rest of the system. Let's be generous ... only about 30 need to be redesigned. There is nothing to help. Refactoring, or even type checking doesn't work across microservice boundaries."
This strikes me as hyperbole. If you take the adage of "team = microservice", you're implying 8 teams manage the payment system. That doesn't smell right.
Secondly, the biggest barrier to adding new fields IME in distributed systems are overly locked down interfaces. If the interfaces allow for extensibility, adding a field, even across eight micro-servcies, takes almost no time at all. I've seen this in a deep system with 4-layer service chain. We had to add a field to the GUI for the next (2 week) iteration, this required prioritization and coordination across 5 teams, one of which was a COBOL/MQ system -- but ultimately took maybe a few days because the interfaces through the chain were designed to be extensible (and the copy book had room for an extra field).
> " Any error you make won't come up unless you're testing multiple of those services simultaneously. If you manage to have just a bit more complexity in your system, those problems won't come out until the massive full-system integration test. You know, the one you can't do yourself, and even your whole department can't do. Oh-oh. That's an awfully long feedback loop to find those 3 dozen places where you forgot to copy that field across."
If the lesson here is that "most organizations are mediocre", you'll get no argument here.
If the lesson is supposed to be "there's a better way", I'm curious what it is. Is compile time type checking and refactoring monolithic software that's managed by 8 different teams is somehow better? I doubt it.
If anything the microservices approach makes it so that each team's service is evolving independently in production, and the system isn't going down or stopping while we all add this field independently across the teams. There is no "massive full system integration test" in the usual sense, there's continuous testing and failure injection. The automated test for adding this field will eventually pass, but there's no stopping the world to wait for it.