Message-Oriented Programming (2017)
joeforshaw.com
joeforshaw.com
I declare my data structures in YAML. From this I generate code for memory allocation; data validation; serialization and de-serialisation routines.
I also declare the topology of the data flow graph in a separate YAML file, decorating each edge with the data type.
Finally, I have a separate YAML file that specifies which computers/processes each node will run on.
Nodes communicate by passing messages. For nodes in the same process, this is simply shared memory (zero overhead). For nodes in different processes that share a memory bus, this is a shared memory queue. For nodes on different machines, the data goes over the network.
Nodes can be python, C or C++.
Very nice. Very declarative. The bare minimum of duplication and boilerplate. Changing from MIL to SIL to HIL is (mostly) just a configuration change. Logging, measurement and replay is built into the framework, as is support for parameter tuning.
Within a single process, the model of computation is synchronous data flow. Across processes, it is asynchronous data flow, but because we have queues with blocking reads and nonblocking writes, we still keep the ability to replay data through the system in a deterministic manner. (Modulo nodes exceeding their time budget).
I'm building something on similar lines with Redis and Protobuf, but I'm in over my head a little bit
- How do you version your messages?
- What problem domain are you working on?
- What kind of process orchestration system do you use?
Process orchestration: Currently very simple. (Python multiprocessing). May change to something more complex in future.
Message versioning: It depends. My configuration management / deployment model means that most of the time backwards compatibility is not required. (The entire system is deployed as a single unit). When messages are persisted, the data recording system files the messages in a configuration management system that knows the system configuration that generated those messages -- so that when they are replayed back -- e.g. to test a new configuration -- the appropriate reading/de-serialization logic can be selected.
It is worth noting: this is part of a custom code-generation/build/test/parameter-tuning system that I developed for embedded machine-vision/sensor-data-processing systems.
It is not intended to be used by third-parties; or distributed as a library -- it is very much integrated into my scripted workflow.
Not the author's fault - he probably just felt the need to share his thoughts on this cool new idea he had.
I just wish HN users would take better care of the implications of clicking the +1 button. You're wanting others to read compelling information you've found.
For me the key here is that it's short, easy to understand, describes an interesting idea (even if I'm sceptical), and avoids trashing other ideas.
I certainly prefer this kind of article to the majority of stories unrelated to tech or business that are very popular on HN.
> I'm sorry that I long ago coined the term "objects" for this topic because it gets many people to focus on the lesser idea. The big idea is "messaging"
http://lists.squeakfoundation.org/pipermail/squeak-dev/1998-...
I definitely liked the idea of tuple spaces but have never seen it implemented in a production system.
Dynamic behaviors are gained at the loss of type safety (function calls are type checked, messages are usually not) and system coherence (just because you can add and remove components on the fly, doesn't mean that it's safe to do so). To compensate for these losses, you need to increase documentation and increase testing. Most people don't and the result is significantly less reliable code.
> Coupling between callers and responders is removed.
But the number of dependencies is the same (reference versus message type). And now it's unregulated dependency without any compiler-level contract between the two. Are the parameters still what you expected? Are the parameters in the correct format? The message type will continue to match, even if these details change.
> Messages can be dispatched to multiple subscribers
Which is great for communication that is intended to be observable but if you're using public message passing for everything, it's going to be tempting to snoop on the message bus and pull data intended for other recipients instead of passing it cleanly. Now your program is a fragile mess because everything is dependent on precise implementation details of unrelated parts of the program.
> Dependencies can be changed on the fly.
Which is another way of saying that you need to handle dependencies changing without notice. What happens if you're expecting a response but the generator of that response is removed between request and sending response? What if you get a timeout? What if you get a partial response?
The compiler guarantees that all sent messages conform to one of a list of possible valid message types. If the subscribers are static, the compiler can also guarantee that every valid message type has at least one handler; if the subscribers are dynamic, you're treading much more dangerous waters but at least you have a comprehensive list of possible messages to verify at runtime that each one has at least one handler.
For the second concern, I agree but I would invert it: if a publisher writes messages intended for a particular recipient, that's a misuse of a message bus. You'd be better off making a regular method call in that case (nobody says the bus should solve every one of your problems). Ditto for the third - if you absolutely need a specific response to your message, don't use a message bus to send it. The challenge of message-bus-based architecture at that point becomes minimizing the number of publishers which need to make direct calls.
In order for your application to do things, certain flows must execute; A is followed by B which is followed by C for some specific functionality.
Without MOP, you might have A calling B, which calls C. This is straightforward to read and debug. If the interfaces between parts are complicated, they may not be easy to test - but this can be fixed.
With MOP, A dumps its output on a bus, and it's picked up by B, which dumps its output on a bus, and it's picked up by C. Now, in order to understand the composition, you need to trace the thread through the entire system, probably using string search - and not be certain you've got it right, because of modalities that may not be explicit.
It looks like the dependencies have been eliminated; but it's a complete illusion. They're are still there, they're just dynamic instead of static. C still needs to follow B which needs to follow A. If any of these bits are missing, the system is broken because it won't be functioning. And even worse, this situation won't be found statically without extra tooling.
MOP and its close relative event-driven programming is effectively COMEFROM as a first-class feature. It promotes write-only programming.
There certainly are reasons to go down that route - for example, you need to execute a lot of unrelated side-effects based on state changes in the system - but don't be under the illusion that it reduces coupling. The coupling is still there because the system breaks when dependencies are absent. It hides essential complexity, not just incidental complexity. Your incidental complexity burden needs to be pretty high before the pros outweigh the cons.
I don't think that's necessarily true. It depends on the implementation. In an actor based system, the syntax might not change that much. You just sprinkle in some async() here and some yield() there. So the code still looks like it is using function calls, and uses the stack/instruction pointer for state management, but is running asynchronously.
I do very much agree with you point about coupling.
First off I have to quibble with his terminology. Usually people say "message" when the sender specifies who will receive the message — perhaps only by a name that is resolved later, but at least that much. What he's talking about sounds more like "events" that are broadcast for anyone to read or ignore as they please.
Anyway, he isn't entirely specific about how he's using these messages, but if he's using a message bus global to the app in order to communicate between different parts of the app, I found it to be tractable and very convenient when writing an Android app. Before I started using the message bus, wiring up connections between different parts of the app was a real pain, and it often had to be redone as functionality changed. When I started routing everything through the message bus, it was marvelously simpler. I just connected everything to the message bus. Obviously that's a recipe for disaster in a large enough system, but I would guess it's a rare app that gets large enough for this to become intractable.
With MOP, A dumps its output on a bus, and it's picked up by B, which dumps its output on a bus, and it's picked up by C. Now, in order to understand the composition, you need to trace the thread through the entire system, probably using string search - and not be certain you've got it right, because of modalities that may not be explicit.
You're right that this is a difficult way to think about it, but there's a much simpler way. You think about elements in isolation and their responsibilities. You have a component that communicates with the back end; one of its responsibilities is to signal when the auth credentials are missing or invalid. You have a component that controls navigation; one of its responsibilities is to pop over a login window and push it onto the navigation stack when the user is prompted to log in. You have a behavioral component whose job is to detect the lack of credentials and signal that the user should be prompted to log in. Each of those components can be tested in isolation, and the only thing that unites them is a list of the messages and their meanings. If the responsibilities of each component are defined in reference to the same set of messages/events then it's clear that if one thing doesn't lead to another then some component isn't fulfilling its responsibilities or the responsibilities haven't been designed correctly.
Again, just as with Redux, I can see managing these interrelationships becoming intractable in a large enough system, but I think it works well for a moderately complex app, and simplifying the plumbing is a huge win.
EDIT: In response to you need to trace the thread through the entire system, probably using string search, if you're working in a statically typed language, you don't need to rely on string search; you can just search for usages of the type of each message. For 90% of the event types you'll find two usages: the place where an event is fired and the place where it's detected and responded to. Most of the rest will be fired or read in multiple places but still a very small number.
I think this gets to the crux of the nature of the solution. You are trading one very strongly statically typed message-sending apparatus (function calls; object member calls) for one more weakly statically typed message-sending apparatus ("events"/"messages" as standalone objects).
While, yes, you can get static typing hints back, for instance, by using statically typed Symbols/Enums for "message types" as opposed to using more stringly typed options, you are still trading very strict static typing (I'm calling this specific function, I'm acting on this specific object) for weaker static typing.
There's nothing wrong with using weaker static typing, of course, and it's nice having a spectrum of options. (It's also why I think it's silly to think its an either/or between OOP and MOP; they are a paradigm spectrum that isn't entirely mutually exclusive and often you want some of both worlds.) It's a trade off, and should be a clear trade off, and knowing that is half the battle.
This is listed under advantages, but I'd say it's a mixed blessing at best.
Objects no longer directly referencing eachother means it is now way harder to reason about and analyze the dependencies in your codebase.
The dependency graph no longer exists in design time like it did before, lending itself to various static code analysis tools that could even draw you a pretty diagram thereof. Now it's grown and shaped in run-time.
Event buses are sort of like a "goto". It's sure convenient to be just sending events back and forth through some static event bus, but once things go south, trying to trace the root cause will turn you into Carrie Mathison.
In Android world, event buses are considered legacy now. One of the most popular choices, Otto, is explicitly deprecated (in favour of RxJava that the authors recommend to use instead).
I'm more familiar with the traditional (SmallTalk-style) MVC setup where the data model entities implement the observer pattern. In that case, observers subscribe to change events by attaching themselves to model values (event sources). I guess MOP differs from this by replacing centralised model value objects with some kind of identifiers used to key message bus subscriptions (?).
Related ideas for client/server are:
Event Sourcing https://martinfowler.com/eaaDev/EventSourcing.html
and
> MOP is a flavour of object-oriented programming (OOP) with the core idea that your objects shouldn’t directly call each other, but communicate by passing messages via a message bus.
Sounds pretty much like a textbook definition of event-drive programming given by someone oblivious to event driven programming.
The benefit is easier multithreading, drawback is memory use (incl. GC churn and bandwidth) and/or complexities of copy on write in multithreaded environment. The style is also not amenable to passing big amounts of data around directly. (That is often worked around via explicit memory sharing or a database - and passing handles instead of data.)
Another name for a different flavour of this is reactive programming.
So, there might be a message queue for each object - This seems to be the way that ROS (the robot operating system) is architected when you have nodes that process messages. Still it becomes difficult to disentagle a datum that needs to be accessed and modified by different queues - and avoiding race conditions, etc. But the solution could use message passing itself. If a queue modified a shared datum, it could message all the other queues using that datum and they could then update themselves. Conceivably, each queue could have its own core and so making parallel processing effortless. You would never need to worry about messing up shared data since the data would look after itself. But the 'problem' is that you have traded time and complexity for storage space - each datum now has a 'train' of addresses it has to visit and it has to carry those addresses with itself.
Event-driven programming has Event handlers (not objects) that pick up specific events. Events aren't messages to a specific Object.
The article seems to be describing a particular style of event-driven architecture, used to organize code within a single process.
What the article states is that multiple sections of the program create messages on a queue, all parts look at the queue and may do something if the messages are addressed to them.
In event-driven programming it is enough to create an event, an event handler, and then pass the event to the handler. Hence, you can define everyhing related to the handling of the events in a single object (event handler).
Although you may also achieve this via implementing a message queue, and multiple event handlers built-in to multiple sections of your code, it is not necessary.
Interfaces and DI are strongly controlled at the composistion root, but it's not easier for highly complex and distributed systems. However the amount of indirection is simpler to reason about and it makes sense for single process systems.
If A send "Process this data X" to B, and B fails with an OutOfMemoryException, it should respond "Unable to process X", or perhaps "Insufficient ressources to process X".
One of the strong points about this way of implementing interprocess communication, is that it allows you to have a new and old version of the same process working concurrently, with the new only running and comparing it's output with the old one. Once you're convinced they work the same, you can retire the old process. To fully leverage this, every message should be platform and implementation agnostic. Even if it's a monolithic system on a single server.
I'm not saying that abstractions are not useful. What I'm saying that over-architecting something on the possibly false premise that "at some time we might need to support X so let's do it by creating an additional abstraction" will waste time, money and will just end up as code clutter which you have to maintain.
- All the programs I work on are running on the JVM so I know how the trace looks like and how it is implemented so it is not leaky nor brittle
-Tracing back what caused what is a pain if you don't see the full trace.
- The indirection caused by an eventbus message passing a microservice boundary makes it practically impossible to debug
- passing on the trace will let you know __immediately__ where the problem is
I'm not a purist, I'm a pragmatist. I don't care if a solution is not pure if it works, spares my time, and most importantly: does not come with a burden. This tracing can be put behind a feature toggle, or an aspect/decorator/whatever.
You might call this a hack, but it lets me ship, maintain and support software while others still stand at the whiteboard planning.
I totally agree that message/event passing is better suited for a complex distributed system
In a more oldschool, bus-less message passing system, different modules can't talk to each other unless they know about each other. That's potential a source of tight coupling if you're binding to specific classes, but it doesn't need to be - you can also couple to a protocol, and be willing to chatter with anyone who speaks that protocol. But still, modules can't talk to other modules unless they're explicitly told about each other.
Pair this approach with an IoC-style architecture where the communication graph must be assembled in a single composition root that lives very close to the application's entry point, and you've got a great tool that gives you an easy way to monitor and manage the complexity of the application.
With a message bus, you're basically just taking that central composition point away and replacing it with a big ¯\_(ツ)_/¯. I've been there, it can be very flexible when you want to build an application quickly, since you don't need to think about how messages will be routed. But I haven't found that the message bus reduces coupling. Since it takes away basically all the need to stop and think about what you're doing before subscribing to a message, you can quickly end up with all sorts of crazy functional dependencies that nobody fully understands because everybody was hacking them together independently.
I guess what I'm cautioning is, don't be too quick to conflate loose coupling with message buses. The one does require that you do the other, but that doesn't mean you can't do the other without the one.
Seems you'll need a consensus layer and the ability to rollback if you don't want to end up in a terrible mess later right?
Regarding the advantages: Interfaces and dependency injection (with or without frameworks) also solve the first two bullet points also rather nicely. And they keep the synchronous nature and strictly typed interfaces. The third point can be achieved with data binding or interfaces which allow for subscriptions (e.g. via classical "Event"s or Observables).
It doesn’t seem that dependency injection really provides the benefits of the second advantage as the author describes it. Specifically he mentions that the recipient can decide wether and/or how to respond to a message. DI doesn’t provide that. The point isn’t _just_ that you can dynamically swap objects, which seems precarious in a lot of ways as values/results in referenced objects could change under your feet which without pervasive usage of observers means lots of easy to make mistakes for devs.
That means for a DI system to really be dynamic in the same sense that you need an event system / observer pattern in addition to DI to provide similar benefits while being more complicated due to now three layers of abstraction (at least) required. Whereas an “MOP" system encourages devs to think about outside changes from the start and makes it the path of least resistance while reusing the same abstraction to enable dynamic target and 1-to-many fanning. The usage of DI systems results in much more ceremony and generally results in overly complicated system architecture as compared to a good messaging system as the author highlights. Now a really complex message topology can become really confusing which results in similar architectural issues as an DI / event based system if not checked.
Classic programming generally doesn't impose restrictions about which direction the data flow is. By default, methods take value from their callers as input, and return values to their callers as output.
An "notification"/"fire-and-forget" interface allowing no data to go back to the caller (no return value, no "output parameter", no visible change to the program state, from the caller's POV) has very interesting characteristics (Like being trivially mockable).
-- I think this is the key cons which might lead us into nightmare.
I imagine the author has techniques for dealing with that kind of problem, but has unfortunately not shared them. Maybe another time.
One of the better Cocoa books motivates it with the following example:
Say you have an app with multiple windows. You also have a settings pane, where you can change the background color of your windows. You don't want the "background color" component to keep track of all of the open windows in the app, that's not at all related to its responsibility. So instead you use the notification center to post a message "hey the user changed the window background color to blue", and windows will subscribe to "the user changed the window background color" messages. This way the color picker and the windows themselves can work together, without ever caring about the others' existence.
https://github.com/greenrobot/EventBus
RxJava is the new hotness at the moment though, which kind of replaces EventBus, so I don't think you could convince a team to develop an app using entirely messages. Still, I'd like to try it for a personal project.
Further, with Win32, you can choose between an async or synchronous message send, depending on whether you are interested in side effects or not.
> It’s harder to reason out program flow
This seems like a big one. You're basically creating a distributed systems problem for yourself.
Object interfaces in static languages are static, most of the time. Thus a lot more of the conversation can be statically analyzed for correctness.
A may be a logger, just track that an X was sent and when. Useful for metric gathering and tracking, but can be disabled (it's unneeded from a functional perspective).
B may contain some actual business logic, take X and transform it into a Y and broadcasting this new message. If we need to change how Y is constructed we can come directly here.
C may use X to update values in a database. Isolating it here allows you to easily replace the DB backend. Create a D process which handles the new database, evaluate that it's working correctly (by comparing to the database C is using) and then disable C later.
These 3 processes are all listening to the same message broadcast. in Erlang we'd need one process which received X and rebroadcasts to an arbitrary number of subscribers (again, easily doable, but not the default behavior people go for). In Go you have channels you can pass around, but as I understand once read the data is removed so only the first subscriber gets it. You'd similarly need some "bus" intermediary that took the input from one channel and rebroadcasts on subscriber channels.
Really, they're describing CORBA and similar systems.
Although I've written buses in the past that let the user address messages to different 'entities' and groups, even ad hoc groups defined by features like spatial locality.