Comment by mixedCase | Hacker News Reader

Comment by mixedCase | Hacker News Reader

Parent Full thread

I think I got it, but I'm inclined to believe I'm missing something.

From what I understood, it boils down to "distributed CPS" (continuation-passing style), where the state instead of being automatically closed over it is explicitly assigned to, and the continuation execution is distributed.

The problem about this specific approachh from where I see it is that something like this should be a few days' work to implement using an off-the shelf persistent queue (which could be abstracted away), something like Protobuf for defining your shared state and auto-generating serializers/deserializers, and then some reusable glue code to push/pull from the queue, and it would support multiple languages almost out of the box.

With that said, I am not sure if I'd want to define pipelines as a shared "mutable" state passed around, it sounds like classic object oriented design which makes it easy to create buggy code. It'd be much more robust to clearly define several steps which very strict, well-defined inputs and outputs every step of the way. You're paying for the copies in each serialization anyway (and much more, since this is a Remote Procedure Call), so what reason could there be for choosing a fixed mutable model?

I think you're onto it, but not quite? The "distributed CPS" explanation is quite good.

There is no "shared mutable state" per se. The "state object" is passed through the stages of the same endpoint, not shared with stages of other invoked endpoints. It is meant to emulate the method local variables that you "get for free" if you code within a method. If you have String firstname set before you invoke HTTP endpoint X, you of course have that firstname available after the HTTP invocation returns, right? This is so obvious that one really doesn't think about it. When you throw a message onto a queue (and then the processor goes back listening on its incoming queue), you of course loose that String firstname, unless you pass it along (But the next service might not need it, so he then just have to pass it on further, so that the downstream service that actually needs it, can get it). But with Mats' state object, if you assigned the firstname to the state object, it is "magically present" on the next stage too.

It is passed within the message, this is what I mean by "messaging with a call stack": The call stack (called MatsTrace) holds the Reply-queues (where an Endpoint's Reply should go to), and the state object which will appear for that replied-to stage. There is no external storage for this.

The point of Mats is to leverage normal developers' innate understanding of normal, straight-down, sequential method-invocation-based coding. You can reason exactly like that, while still being able to code fully async messaging.

I love the Loom project, and I feel that there is a massive comparison here: Instead of having to embrace async/await, or Promises, or whatnots, Loom instead "hacks" the threading model. The result is exactly the same as if you were using async/await, but with a bit more overhead, you can code linearly, not having to warp your brain around the mess of async/await/Promises. This is exactly the same rationale for me making Mats.

Reply on news.ycombinator.com