stolsvik

That anyone with more than 6 months of experience still drag Hibernate up from their chest is just absolutely beyond my comprehension.

Is it only me that sees a problem here?

Mats3’s tagline: “Message-Oriented Async RPC. Message-based Interservice Communication made easy! Naturally resilient and highly available microservices, with great DevX and OpsX.”

JBang is a cool "execute single Java source file with dependencies" solution, which offers a simple way to test new libraries. The article tries to showcase Mats3's core concepts through a series of JBang scripts.

To streamline the process, a small library called 'MatsJbangKit' has been developed, which not only takes care of pulling up the Mats infrastructure but also depends on all the necessary dependencies. This means users only need to reference this single library in their JBang scripts. The aim is to pique curiosity and inspire further exploration of Mats3 and its potential.

If you like it, a star on Github would be much appreciated: https://github.com/centiservice/mats3 (So that buying stars won't be necessary! https://news.ycombinator.com/item?id=35207020)

There's a previous discussion of Mats3 here: https://news.ycombinator.com/item?id=34765578

This is pretty good: “what is mats3”, and it strings together pieces of the webpages, and summarizes in a way that makes quite good sense.

Monitor by having some health check evaluating that no row stays in PROCESSING for too long, and that no row stays NOT_STARTED for too long, etc. Introspect by making a nice little HTML screen that shows this work queue and its states.

As I wrote in another comment, this is somewhat similar to a "work queue pattern" I've described here: https://mats3.io/patterns/work-queues/

I've written about a somewhat similar problem in context of the Mats3 library I've made: https://mats3.io/patterns/work-queues/

And yes, the point is then to use a database to hold the work, dispatching from that. In the Mats3 context I describe, the reason is to pace the dispatching to a queue (!), but for them, it should be to just run the work from that table. Also, the introspection/monitoring argument presented should be relevant for them.

That a message queue library fetches more messages than the one it is working on is totally normal: ActiveMQ per default uses a prefetch of 1000 for queues, and Short.MAX_VALUE-1 (!) for topics. The broker will backfill you when you've depleted half of that. This is obviously to gain speed, so that once you've finished with one message, you already have another available, not needing to go back to the broker with both network and processing latencies: https://activemq.apache.org/what-is-the-prefetch-limit-for

In summary, I feel that the use case they have, "thousands of jobs per day", which is extremely little for a message queue, where many of these jobs are hours-long, is .. well .. not optimal use case for a MQ. It is the wrong tool for the job, and just adds complexity.

That is interesting - this is actually a feature of JMS, which is employed in Mats: You can say that a Mats Flow is nonPersistent (i.e. "not all that important, really"), and in the same call you can also say how many seconds until timeout.

<https://mats3.io/javadoc/mats3/0.19/api/io/mats3/MatsInitiat...>

This is employed by the MatsFuturizer if you use the "nonEssential" invocation: <https://mats3.io/javadoc/mats3/0.19/modern/io/mats3/util/Mat...>

The point is that if there are too big queues, bunching up even more work on them which will not be processed in time does not make any sense: If the Future on the caller side have already timed out, it makes no sense at all that the callee should process the work and return the answer.

Wrt. returning a Future, this is what the MatsFuturizer does, giving you a sync-async bridge into the "Mats fabric": https://mats3.io/docs/sync-async-bridge/

You shall obviously set up monitoring of the MQ and all its DLQs. Once you've done that, you will literally never have a failing processing flow that isn't caught. A successfully initiated Mats Flow will either run to completion, or it will DLQ. Guaranteed.

The MatsBrokerMonitor is a Mats-specific such monitor (compared to the generic monitor ActiveMQ comes with). The pages here are a bit lacking, but it at least explains a bit: https://mats3.io/docs/matsbrokermonitor/

Once you're in, you only refer to the queue names. Which in Mats effectively are direct references to the endpoint name, typically in a Java-style class.method way, e.g. "OrderService.placeOrder".

It is each stage that is transactional. If the stage processing fails (or the node crashes), both the DB transactiona, and the messaging transaction, is rolled back.

It is then retried. ActiveMQ has a default of 1 delivery, and 6 redelivery attempts. If those 7 fails, the message is assumed "poison", and is put on the DLQ.

But this means that a Mats Flow is a series of transactions, each is individually handled. As you probably allude to, you cannot roll back the entire flow - it is just the particular step that is rolled back. Thus, if the message ends up on the DLQ, you have a mid-way process, where the steps in front are already done and committed, while this stage, and any downstream, are not yet done.

The message is however on the DLQ. If the problem was e.g. an temporary database failure, which now is resolved, you can now just reissue the DLQ (move it from the DLQ back to its queue), and the process will continue as if nothing had happened.

Just browsed it again, and there seems to be a bit more tooling in place. But I fail to see something like Mats in its feature list, though.

Mats is about endpoints which can invoke other service's endpoints. It is "messaging with a call stack", simplifying how you can code messaging-based inter-service communications. Pub/sub is not the same, and (synchronous) REST/gRPC is what Mats explicitly is an alternative to.

This is not how it works. A messaging broker client connects to the message broker. The client then creates receivers for one or multiple queues. If the broker has multiple clients receiving from the same queue, it uses a round-robin dispatch to the clients.

I fail to see how DNS or config, or "reliably add and remove" factors in here?

> Wanna bet it's a built in feature in your messaging system?

Yes?

This is not using pub/sub, but queuing. (There is an option of using publish/subscribe, but that is only meant for special cases like updating a GUI, or "broadcasting" an event like "invalidate cache for user X")

Message Queuing can be transactional. Mats leverages this. I have written about it multiple places both on this HN discussion, and on the webpage.

There is no "shared mutable state" per se. The "state object" is passed through the stages of the same endpoint, not shared with stages of other invoked endpoints. It is meant to emulate the method local variables that you "get for free" if you code within a method. If you have String firstname set before you invoke HTTP endpoint X, you of course have that firstname available after the HTTP invocation returns, right? This is so obvious that one really doesn't think about it. When you throw a message onto a queue (and then the processor goes back listening on its incoming queue), you of course loose that String firstname, unless you pass it along (But the next service might not need it, so he then just have to pass it on further, so that the downstream service that actually needs it, can get it). But with Mats' state object, if you assigned the firstname to the state object, it is "magically present" on the next stage too.

It is passed within the message, this is what I mean by "messaging with a call stack": The call stack (called MatsTrace) holds the Reply-queues (where an Endpoint's Reply should go to), and the state object which will appear for that replied-to stage. There is no external storage for this.

The point of Mats is to leverage normal developers' innate understanding of normal, straight-down, sequential method-invocation-based coding. You can reason exactly like that, while still being able to code fully async messaging.

I love the Loom project, and I feel that there is a massive comparison here: Instead of having to embrace async/await, or Promises, or whatnots, Loom instead "hacks" the threading model. The result is exactly the same as if you were using async/await, but with a bit more overhead, you can code linearly, not having to warp your brain around the mess of async/await/Promises. This is exactly the same rationale for me making Mats.

If Stage N in the total process has picked up a message, starts to process it, and then something (temporarily) fails (or the node crashes), then it will roll back whatever DB operations you have done up till then, and roll back that it has picked up the message.

The MQ will now reissue the same message, and it will be picked up again. This time, things work out, a new message is produced, and the entire processing of this stage is committed.

So, either you have received a message, done your DB-stuff, and sent an outgoing message, or you have done none of that.

I do not see you easily do that with REST. Or at least you will have to code quite a bit to get such nice semantics. With messaging and Mats, you get it entirely for free.

To be fair, it is not entirely true. There are two separate transactions going here, and it can fail in an annoying way. I write about this here: https://mats3.io/using-mats/transactions-and-redelivery/

I believe this holds for very many business processes. If you get a new shipment of widgets, and you can now fulfill your orders waiting for those widgets, it does not really matter if it takes 1 second, or in a very bad outlier day, occasionally 2 hours.

Realize that the point is that this settling, or order fulfillment, will go as fast as possible. Usually within seconds or maybe minutes. However, if you suddenly get a large influx, or the database goes down for a few minutes, this will only lead to a delay - there is nothing you need code up extra to handle such problems. Also, you can scale this very simply, based on what holds you back (services, or database, or other external systems, or IO). It will not be the messaging by itself!