MQTT turns 25 | Hacker News Reader

MQTT turns 25

andypiper.co.uk

289 pointsandypiper74 comments

I think my first project that's still "in production" and used daily was taking an existing SVG map of a water distribution system (snowmaking and fire suppression pipelines/pumps/valves/etc at a major ski resort). I made a MQTT topic for each pump, valve and section of pipe along with some state for each (water direction, pump or valve on/off, pressures, etc). It's a website, so I used mqtt.js and jQuery to update the colours and fills of the SVG to represent the various states. It's statically hosted and there's a mqtt broker that's been running in a container, untouched for nearly 10 years now. Mqtt.js is running over a web socket, so if the state is updated, it just magically updates for everyone else.

Holy shit Docker is 11 years old.

It was early days, yeah. The other part of the project that's been depreciated now was running node 0.4, if I am not mistaken. The watermap was part of a bigger asset tracking system, node + jade templating (before jade became pug, even). The watermap was actually embedded in a jade template, but when I depreciated the other system, I ended up just copy and pasting the output from the jade file into an index.html, and everything just kept working :D

aaah I love me some AJAM ;)

Having recently used MQTT for a project, I can't say that I'm a huge fan. There are a lot of options in the protocol where it's not immediately clear what they do and why they are important, or in what combination they need to be used to make sure things work as intended, and the documentation is often not great at explaining. Part of this may also be on the Eclipse Mosquitto Python client that I had been using. I took me several days to figure out that the client was running into a race condition on a slower system that resulted in subscriptions to topics silently being ignored which caused the associated callbacks to malfunction despite following their documentation 100%. Overall, this has been one the messiest experienced I have ever had with a protocol, certainly one that isn't that old.

My experience using MQTT has been through the paho python MQTT library, and, while we definitely have managed to get a lot done, it has been a terrible experience.

Everything about it rubs me the wrong way. The API design, the poor documentation, even the way they seem to not adhere to typical python conventions.

It looks like it's easy to start using it, but then the breadth of protocols and implementations start getting to you. At a point in time, I remember the only reliable way we had of knowing if we had successfully connected to a server was to try to subscribe to a topic twice and catch a specific error code on the on_connect message (which actually was documented as a success code at the time). I know how crazy it sounds and maybe there was a better way to do it, but, if there was, I don't think it was that easy to find.

It's easy to complain, though. I'm very grateful so many people have worked to create this library. If they hadn't, I wouldn't have built what we've built with it. I really admire people who take on these huge projects.

So your client just sucks here.

My experience with the eclipse clients (paho, etc) is similar to yours in both python and C++, - it makes it overly complicated and seems very low level. It also is somewhat buggy because of the architecture.

I believe (perhaps wrongly), it's all basically maintained by one person or close to it (only one person has contributed to the C++ client in the past 6 months, for example, and nobody has contributed in the past 3), so i kind of understand how it's gotten this way over time.

I filed a simple PR (1 word change) to fix an obvious bug and it took 2 years to review it and accept it with no changes. Again not a complaint, just trying to portray how the state feels - a lot of libraries, bugs, and work, and few overworked people helping get it all done.

I moved to other clients and the experience is much much better in python, rust, C#, and C++.

Most of them have a good combination of high and low level API's, so if you just want to send a message on a topic, you don't have to worry about acks, retries, etc.

But if you need control, you can get it.

Honestly, i worry at this point that keeping paho/etc alive in this state is doing more harm than good - if they were officially dead it would at least force the issue. Right now you end up with users who have an experience like yours, and then either give up on MQTT or assume they are doing it wrong :)

After reading your post, I went looking for good C/C++ implementations suitable for embedded development, and honestly, I couldn’t find many! MQTT is typically straightforward to implement, so most companies develop their own version to better adapt to the specific systems they use for data transmission. That said, maybe this is an opportunity to create a well-designed MQTT client implementation tailored specifically for embedded devices! :)

I work for the team that maintains coreMQTT (https://github.com/FreeRTOS/coreMQTT)

Its a C89 MQTT library intended for embedded devices.

There is a very nice MQTT client C++ implementation that works great on embedded devices too. As it so happens, it is right now under formal review for inclusion into Boost. It is called async-mqtt5 and can be found at: https://github.com/mireo/async-mqtt5/

Could you list the clients you are using? Because like many I fell into paho and things are not great.

I've used the Paho APIs in Python, C and C++ and moved on to not using any of these APIs. Where possible I use mosquitto_sub and mosquitto_pub to support the protocol and then just read/write standard input/output. It's not a matter of bugs (though I have encountered at least one) but rather it's just easier to use a program that's been written and tested to manage the connection to the broker.

The only place where I was unable to do this effectively was with the last will and testament message (and I might not have tried hard enough.) It also doesn't work with microcontrollers that don't run something like Linux.

The eclipse paho MQTT Java client is also terrible, lots of open bugs, terrible API, and no release in over 4 years. For my case, I was able to use NATS instead.

https://github.com/eclipse/paho.mqtt.java/issues

While not directly competing with mqtt, https://pipe.pico.sh is a great pubsub tool that leverages ssh for communication, effectively creating an authenticated, networked *nix pipe system.

It tries to be the simplest way to send and receive events.

> There are a lot of options in the protocol where it's not immediately clear what they do and why they are important

I totally disagree, your expectations for the protocol seems unusual. They are not frameworks, and need to game many options.

MQTT specification is like 3 pages.

For comparison try to read specification of HTTP3, or AMQP - those are hideously complicated.

Also, they are options. Basic pub/sub usage doesn't need any of them, though some are pretty nice to have.

I read someone somewhere saying "why not just use TCP" and for my IOT project plain TCP was fine. MQTT didn't have infinite queuing which was important for my use case, and if I'm going to have to track message ids to feed into MQTT which tracks its own message ids to send to the broker, why use it at all.

I'm not sure what the sweet spot is, unless it's just interfacing with projects that have already committed to MQTT.

MQTT is pub/sub protocol with a concept of brokers and clients. It's not in anyway comparable to TCP. You could rewrite an equivalent of MQTT on top of TCP in a fairly straightforward way - MQTT is simple by design - but I think you take the question from the wrong end. Why would you write a new protocol when MQTT already exists and does the job?

The effort to vet MQTT providers, integrate them, and configure everything properly, understand and work around MQTT and implementation quirks, and the protocol overhead were greater than doing something directly on top of TCP, and I don't believe I lost any features that I needed.

MQTT does have its nuances, and diving into the specifications is sometimes required to grasp all the options fully. That said, for most cases, simply focusing on the API for the library you're using should suffice. It sounds like the real issue you're facing is more with the library than the protocol itself.

If you're open to using Lua, you might find this client library more straightforward: https://realtimelogic.com/ba/doc/en/lua/MQTT.html. It’s designed to be simple to use and works with the BAS runtime, which supports many platforms, including microcontrollers like the ESP32: https://realtimelogic.com/downloads/bas/ESP32/

MQTT is being used a lot more in recent years inside of factories for sharing data between machines. Historically it's been used in Oil & Gas for SCADA (getting data from remote well sites).

10+ years ago we added it to Kepware (OPC server) and streamed tag values to "the cloud". I was at a conference giving a presentation on it when Arlen Nipper, one of the creators of MQTT, came up after the presentation and said I did a "decent job". It was humbling :). Fast forward to today, and we have a new company (HighByte) modeling factory data at the edge and sending it via MQTT, SparkplugB (protocol over MQTT), direct to S3, Azure Blob, etc, etc.

All that to say, MQTT is a big driver in Industry 4.0, and it's cool to see it so heavily used all these years later.

Is MQTT being used in places where previously Modbus would have been used?

In some cases yes, controllers are adding in MQTT support directly or SparkplugB.

It works in simple cases, but the lack of transactions & ability to control what and how often you're reading from the client side, can create challenges.

There is also MQTT integrating bus-hardware- you basically plug in as bus-hardware replacement which allows to send data directly from the bus to the broker. The PLC never know what hit it.

https://www.ifm.com/de/de/product/AL1332

Pretty wild times.

PS: You can web GET/POST set and get ios with these. Robot experience that tragic irony POST {NOOO} and he just dropped it like its hot.

I also work in that industry and am quite puzzled by the technological inferiority, scope creep and NIH syndrome of the OPC foundation’s standards. I wish they just used sparkplug b and implemented their specs on top of that for semantics.

Then again the async stuff they’re doing now is so over-engineered and terrible as well.. I took part in their teams meetings some time, only to find they had never read the mqtt spec (less than 50 easy to read pages) and didn’t understand what the headers are for for example (they wanted to put stuff in there that actually belongs in the payload). Some microsoft guy also took offense at my suggestion to first look at what competitors are doing with mqtt, because he wanted to create something new rather than copy. My company will just treat opc ua at the very edge and isolate it from our tech as much as possible on my suggestion.

Umm. There is a deep integration between OPC UA and MQTT since ca. 2017.

For binary and JSON payloads, and with encryption and group key management features.

I see quite a bit of unsubstantiated bashing lately. Especially advocating 20% solutions that solve the easy parts with less effort.

What are you missing from OPC UA?

We are currently using the Kepware IoT plugin to stream around 800k tags per second via MQTT ultimately into a Victoria metrics DB. It's a little clunky and requires more processing steps than I would like. I am currently moving away from this solution (since Kepware charges a reoccurring license fee every year for the IoT plugin), and toward using telegraf to read the OPC-UA data directly from kepware without the IoT plugin.

Do/Did you work at Kepware?

> MQTT is being used a lot more in recent years inside of factories for sharing data between machines. Historically it's been used in Oil & Gas for SCADA (getting data from remote well sites).

This is where I first encountered it too -- chemical production. Also saw a lot of it in aviation and railroad control systems.

That said, seeing a lot more Kafka and RabbitMQ crowding into MQTT's marketspace.

About 15 years ago, Andy Stanford Clark's house became something of a news item, in the days before tweeting IoT devices were a thing:

https://www.bbc.co.uk/blogs/technology/2009/06/things_that_t...

The core protocol, devised in the days when 1 byte over a satellite link cost $1, is incredibly efficient and simple to implement.

Reposting my comment from five years ago:

I once had a client where the only port available for me to use on their firewalls was for MQTT (1883) because that's how we were getting sensor data from them. They would not open anything else for us no matter how we implored them so I wrote a live TCP wrapper over MQTT to get around it. It was a local multithreaded TCP daemon that listened for outbound requests on a certain port, wrapped them in MQTT and then published them using a unique topic. The server daemon would detect these topics and unwrap them before forwarding to our server processes. So the client machine thought it was making a live TCP connection to our server but in the middle was a funky invisible MQTT wrapper. It was really elegant once it worked but my goodness was it a pain to debug - a couple of months before I got the whole thing right because of all the blind alleys I went down.

I’ve worked at places where subverting firewall rules like this would get you fired. I now refer to such situations where I’m forced to twiddle my thumbs while no work gets done as “letting the process work”.

I always kick this kind of decision upwards: we can do it this way and break this rule, so I need director level approval, or we can wait until the customer blinks.

Yes. Ask your boss for approval and tell them otherwise nothing will get done.

So you basically implemented an MQTT-Sockets protocol. Which you could then use to connect to a WebSockets server on the other side.

I'm trying to remember the reason we couldn't use MQTT-Sockets. I do remember we couldn't though. Also we could not use Web sockets server side.

Fun fact: at the same time, the most famous C++ library, Boost, is reviewing the `async-mqtt5` implementation (https://github.com/mireo/async-mqtt5) to be included in Boost as Boost.MQTT: https://lists.boost.org/Archives/boost/2024/10/index.php

Is Boost still something people reach for in newer projects? Anecdotally most adoption I've seen happened in the 00s and very early 2010s (pre everyone mandating C++0x/C++11), I only rarely see it around these days.

edit: boost.org is a blast from the past, still looks exactly like I remember it from like 2008. Down to the "Get Boost" shopped on an emergency off button!

That is an interesting question. Here you can see the actual popularity of each library within Boost: https://grafikrobot.github.io/boost_lib_stats/. Also, see the comments at https://www.reddit.com/r/cpp/comments/130bzj8/has_boost_lost.... My company has been heavily utilizing Boost.Asio and Boost.Beast, so we've definitely reached for Boost in newer projects as well.

AFAICT Boost.Asio is still the go-to networking library. (Though Asio is also available as a standalone outside of Boost, I didn't know that until recently.)

Apart from that, Boost has quite a few goodies such as Boost.Json, Boost.Program_options, Boost.Interprocess, Boost.Lockfree, Boost.Unordered, Boost.Dynamic_Bitset, etc.

Some libraries, like Boost.Atomic, Boost.Thread, Boost.Chrono and Boost.Filesystem, simply became obsolete with modern C++ versions. In fact, they served as the blueprint for the corresponding C++ standard libraries.

Personally, I have been wary of using Boost in the past because it's such huge library, but CMake integration is actually quite good these days and I found it pretty easy to use. Documentation is also quite good IMO.

Boost appears to have become some kind of staging area for stdlib. Stuff that becomes popular in boost get eventually included in stdlib.

It's such a great little protocol, and besides being "small enough" for hobby projects, it also scales enough to be used for something like Facebook Messenger [1]

[1]: https://engineering.fb.com/2011/08/12/android/building-faceb...

I don't buy the b.s. around MQTT being so lightweight/efficient/etc. It's just using TCP/IP. Maybe by the standards of the day that was comparatively special, but I've never seen any real evidence for the claim besides this constant bragging.

I guess it's nice that MQTT is a standard and so you have a way to connect to off-the-shelf devices that are enabled. I still think there are better options for pub/sub and message queues, esp. if you need to failover on the consumer side.

What better options? The neat thing about MQTT, and that almost all other pub/sub implementations that I've seen get wrong, is that the core data structure in MQTT is the subscribing client, not the queues & topics. That means you can map as large an address space as you want into the topic tree. The topic tree can be trillions and trillions of endpoints, even an embedded server could have a topic tree with one endpoint for every ipv6 address, if you wanted. So the topic tree is rich, your subscriptions can be as selective as you need, and the server can be fast on a low footprint.

Exactly. Even shorter summary: MQTT broker is stateless thus easy to manage and hard to break. This makes MQTT reliable.

> I still think there are better options for pub/sub and message queues, esp. if you need to failover on the consumer side.

Sincerely, what are these options?

The various message-queueing products, Kafka, et al. Sorry for late response and not responding sooner...

I have been using MQTT for years in my IoT courses – it has proven to be a very versatile tool, and it is convenient that it is also supported over web sockets.

I had a lot of fun on an embedded systems project recently where I used MQTT as an interprocess messaging system. Broker and clients on the same machine.

If I needed to sniff or debug something it was as easy as putting the device on the network and using MQTT Explorer to record or inject messages. I could even let coworkers that were working remotely work with the system by opening the port outside the LAN.

Haven’t seen this done much but it does seem to have some desirable properties. My biggest concern when using it as a system component has been durability guarantees. I don’t have a lot of trust on broker implementations not to lose data.

> My biggest concern when using it as a system component has been durability guarantees. I don’t have a lot of trust on broker implementations not to lose data

(If we ignore retained messages) MQTT brokers are not designed to store data, so there's no need to be concerned about durability because there simply isn't any...(!)

It is (just) a messaging protocol.

MQTT, by design, isn't going to have the confirmation mechanisms and safeguards that you could get from other methods or even sockets. But once you release yourself from that paranoia, it's pretty liberating.

The nature of my project was that a missed message wouldn't be fatal, and message retention was good enough protection for a process that crashed and restarted.

Would zeromq be an alternative here?

It's an alternative, but there are some aspects to having the broker that I liked. If there was a bug I could halt the system, attach/subscribe an external dev machine, and grab all retained messages from the broker. I don't believe ZMQ allows that.

We use ZeroMQ for this purpose. No broker needed.

this statement is a bit confusing. because with zeromq, one side plays the broker (I dont mean the broker pattern). I mean it in the sense that someone, somewhere, needs to listen on a port. which in the end is the same as having a central mqtt broker sitting somewhere listening on a port. hence the statement "no broker needed" is a bit confusing.

> one side plays the broker

Hopefully most people understand that I meant that you don’t need a stand-alone broker, which on embedded systems means one more thing to deploy and worry about the license for.

Edit: Also:

> The philosophy of ZeroMQ starts with the zero. The zero is for zero broker (ZeroMQ is brokerless)

https://zeromq.org/get-started/

I think you can also connect in a random order, so bind doesn't need to happen before connect.

The cool thing about MQTT is there's so many servers for it that are completely public -- and intentionally so. You can use it for real time messages but I think it also has quality modes that allow for some persistence.

Some of these servers have stat pages that show how much data they transfer. Usually phenomenal amounts of data. It's pleasing to me as someone doing a lot of stuff with open source networking to have such an establish protocol (instead of say -- 10 crappy ones.) MQTT works and it works well.

Have any examples of public servers?

HiveMQ has a public one: https://broker.hivemq.com/

I only know this one

https://www.mqtt-dashboard.com/

But a quick Google search will get you some others.

Helsinki Regional Transport (HSL) has a public server with realtime position of all of their vehicles (buses, trains).

https://digitransit.fi/en/developers/apis/4-realtime-api/veh...

I discovered MQTT with Home Assistant (a home automation solution, no, sorry, THE home automation solution).

It helped me to understand pubsub philosophy and I ended up using it for my personal development - to the point where I was misusing it (which is the typical path with technologies one discovers)

It is a great protocol for home automation but quickly gets annoying for more complex scenarios.

There aren’t many telemetry transfer protocols suitable for IoT besides MQTT (especially for resource constrained systems). It is incredibly simple to implement, has numerous libraries available in various programming languages, is supported by major cloud providers and is designed to operate over TCP (meaning you avoid DTLS).

We have shipped many products that use MQTT for telemetry transfer to various different vendor servers and it always just worked. It was never a pain-point in stark contrast to CoAP or some proprietary protocols.

My most recent foray into MQTT was using the MQTT 3.1.1 compatibility mode in NATS, which was pretty easy to set up once I got the permissions sorted out.

Works great for my smart home needs while also letting me explore latency and throughput load testing if I want to have a little fun.

I'm using NATS as well, natively when possible but bridged to MQTT for devices that need it.

I remember one of the annoying things I found about actually using MQTT was that many clients didn't implement any sort of connect/disconnect callback and expected you to handle reconnection logic (e.g. exponential backoff) yourself. Also it didn't support RPC style requests for some reason.

Also since MQTT is TCP-based you can't natively speak the protocol from a webbrowser without the broker also listening on a websocket.

Ended up switching to WAMP-proto [0], which checked all my boxes and just did everything over websockets by default.

0: https://wamp-proto.org/

MQTTv5 now has RPC. You can specify a response topic in the request message, and also add 'correlation' data to track the context of the call.

Most of my home automations depend on it, it’s been great honestly. Cheers to mqtt!

I'm wondering if someone tried https://mqtt-udp.readthedocs.io/en/latest/

Sounds like a very good idea on paper, which can even be brokered if needed

A heap of dead "replacements" at the feets - what more can a communication standard want.

I never understood why we don't use MQTT for simple systems monitoring, with a push model.

I mean it was almost designed for this use case.

Also if using NATS things become very simple to setup.

What do you think about LwM2M in comparison with MQTT?

It is not directly comparable because LwM2M was originally developed for device management, while MQTT is primarily used for transferring telemetry or other opaque data.

My personal gripe with LwM2M is that it uses CoAP (Constrained Application Protocol) as its primary transport protocol. Since CoAP is based on UDP, you’re forced to rely on DTLS for authentication and encryption. Managing sessions with DTLS can be quite frustrating, and unlike TCP, there’s no reliable indication of when a session is closed by your peer.

Another issue is that it doesn't strictly follow the client-initiated communication pattern and can receive unsolicited messages from the server. I have found this to be very unreliable in real-world conditions for Device Management use cases.

Reply on news.ycombinator.com