On Monoliths and Microservices

I've built a couple of systems using this method in my early days of attempting to smash large systems down into smaller ones (around 2006-2011), and explicitly using ESIs and then Varnish ESIs as the layer of orchestration and integration.

There used to be some real drawbacks to this, especially as Varnish wasn't really happy with potentially hundreds of ESI calls per HTTP request, yet that was what we were doing (a call to a JSON API that returned a collection of things would create a document that contained ESIs pointing to each thing, and each thing could also contain ESIs to hydrate themselves, and so on).

One of the interesting bugs was that if you had a lot of ESIs then only the first 10 or 20 succeeded, and the rest did not. We raised bugs for this at the time (Varnish 2.1.4) and they were all resolved (by Varnish 3).

Varnish is great at this now and this approach can and does work, but it does still require discipline to ensure that other pitfalls are avoided (circular references, how to handle errors if you are serving non text/html, how to prevent putting a strain on Varnish now that you're forcing it to orchestrate and manage connections for all subresources).

But yeah, it works.

However... I personally moved away from building this kind of thing as it really wasn't an elegant solution.

This approach made it more difficult to add things like tracing, soak testing of new endpoints, and many other things that really increase in importance after you've gone through the first iterations of many simple v1 services.