XMPP: A Communication Protocol for the Internet of Things
briskbraintech.com
briskbraintech.com
The idea behind federation with XMPP for humans (aka, Jabber) is that my friend can have an account on server A (friend@a.com), and I can have an account on server B (me@b.com), and we can tell each other our addresses (because we know each other in real life), and then get messages to each other, which is cool, because we're friends, and both know English, and so will be able to meaningfully communicate. Not bad!
The idea behind federation with XMPP for IoT devices is...what? That my smart lighbulb can be talking to a control server run by Phillips, and my smart fridge will be talking to a server run by Samsung, and...they have no idea each other exists, have no ability to address each other, and have no ability to generate messages the other can consume, because they're completely different devices, from different manufacturers, with different protocols? Even if the fridge wanted to tell the smart lightbulb to turn on when the door is open, they need a shared "lightbulb control protocol", no? This feels like claiming email lets any two people on Earth hold a meaningful conversation; just because you can transport the message to the recipient doesn't mean they can understand the language of the message.
How does federation help here? Or any other XMPP feature? There's just no attempt to tie any XMPP feature or characteristic to the nominal use case.
IOT vendors have a less clear use case and they've shown they're not particularly interested in openness or compatibility (or security). Why would this be any more likely to succeed for IOT than it did in messaging?
You're right that it has to start with sending a message, but there's no incentive for companies in this industry to ever take step 2 here.
If you have a billion “users” using a working chat system and the concern is “with federation they can chat with outsiders and we can bleed users” you don’t just drop the whole thing, you turn off federation and prevent third party client logins.
It'd be cool if the sunroof can open when it gets the message from the weather monitor that it's a nice day out, and the AC can switch off when it gets the message that the sunroof is opening...
Heck, the best way for your house is to use multicast, with a unique IP for each endpoint type, and a unique port for each distinct endpoint within the type.
The endpoint IDs could even be self assigned, with the idea that you’d only really need a gateway for use outside the home. Standard but extendable protocols can be developed too, so everything can interop but still include custom stuff beyond the standard.
XEP-0174 is a peer-to-peer extension to XMPP which uses mDNS ("Bonjour") to discover other clients on a local network. For a while, Apple had an implementation of this in iChat.
It's highly unlikely to pan out in the real world, there's a lot of assumptions there that Philips, Samsung + co are highly unlikely to satisfy.
And of course, you can get notifications from all these devices right in your chat.
Yeah, but they haven't. And almost certainly won't. And even if they did, in order to interact with each other they'll need to agree on a common language, which would be an enormous task.
But if that enormous challenge was solved, then yes, federation would be one of the ways you could let devices find each other, but there's others. For example, the home router you suggest could configure all your smart devices could also, say, act as a simple registry.
Saying "federation does help" oversells it greatly, I think. It's more like "federation could help, one day in the far future, if we solve a lot of difficult problems".
That is what a message broker [1] is for.
Say you introduce a new device to the network. Whatever magic transport protocol you are using discovers it and your control hub takes ownership after some handshake process. Now, how does the hub—and by implication every other device on the network—know what that new device is and what it can do? It’s a lightbulb? Okay, that’s just on and off right? Oh, no it might have dimming capability. Maybe it’s a candelabra with multiple bulbs? Maybe the bulb can show different colors. Maybe it has a wake-up gradual brightening process built in. Maybe it has a sensor built in to adjust brightness and color temperature based on existing light sources. Maybe you want to disable that but also set the color to purple. How do you exchange all this capability information? And this is just a lightbulb.
XMPP federation doesn’t solve for that.
It's a sad realization, I always thought XMPP would save the day some day, but apparently there's a lot of work between now and there. And it might just as well be called something else. I think the latest move towards HTTP/3 enforcing SSL1.3 is one way to envision a fully secured web, 10 years from now.
It seems to me that this is basically describing Matrix [1]. Funnily enough, since its current transport is HTTP, it looks like by the time HTTP/3 becomes a thing, network hopping would be solved essentially for free.
[1]: https://matrix.org
I’m most likely either misunderstanding what you mean by hopping or completely missing a feature if quic.
Thank you
Even most barebone request often eats more bits than data
That's why nor HTTP nor XMPP will be going anywhere
We were blowing through our 1MB (yes...) SIM cards quite quickly, and it was 98% just packet overhead. I can only imagine what it would have been like over http.
It is true, the protocol was invented in 1999 for then-relevant conditions. There are good things - everything that is related to federation works remarkably well, and bad things - multi-device usage, reconnection, etc.
However, the very thing that is mostly panned here by Matrix proponents - modularity and extensibility - allows to uplift it for the modern world. Actually, we recently did just that, by developing a bunch of new protocols and implementing them on an XMPP server. I'll spare the verbose details, but in short, we redesigned reconnection process from a resumption of an XML stream to restoration of messenger state, and it works rather well. That set of protocol extensions greatly helps to improve the performance of our upcoming XMPP client.
Our protocols are public, and we plan to submit them to XSF, once we're done translating them from our native language. Hopefully, these XEPs would be supported one day by other clients. We believe they are crucial for XMPP future.
It's been about a decade since I had the oh-so-common billion-dollar idea (read: extremely costly but nowhere near profitable) of "hey, how about we used private servers/clients in peer to peer to communicate? Like MSN and descendants but without a central server?" I think me and the 100,000 others with the same idea felt the challenge of improving XMPP + working on conversion from existing platforms was too tall an order.
Now, if guys like you have been/are improving XMPP 'enough', then I can see a path for the open ecosystem to grow anew. And that, provided alternatives like Matrix etc. provide a path for data (API, URLs etc), is just what we need to massively disrupt the basic communication space.
Really, thanks. Spark re-ignited for social apps and a new social paradigm fitting our times.
I have written a lengthy article in 2016 in the state of mobile XMPP: https://gultsch.de/xmpp_2016.html
The situation has only improved from there.
You know, I like the sound of you and the makers of Xabber. The "myth" of XMPP being unsuited to mobile / IM is clearly overblown if you're only even half-right about it all (because then that means there's no "fundamentally broken" paradigm). It's great to hear the community is active nonetheless! ~
I started reverse engineering it but haven't quite finished. The plan is to connect it to Home Assistant and negate any need to communicate outside my house. To get the above output, I wrote a tool for logging the communication in iOS: https://github.com/brianhenryie/xmppframeworklogger
Regarding the use of XMPP for it, as pointed in other comments, it indeed looks like unnecessary overhead, but I think it is quite similar to using HTTP for transfer of things other than hypertext: not a perfect fit, but already has good software support, standards, and additional features that are useful. And perhaps more importantly, no wheels to reinvent poorly and hide from customers.
[0] https://xmpp.org/extensions/xep-0323.html
[1] https://xmpp.org/extensions/xep-0324.html
If you want to IoT, why not just use NATS or something?
As an aside, this article seems quite devoid of actual content, bordering on spam IMHO. The title mentions IoT but the term appears only once in the entire article at the end, in a vague "XMPP is for messaging and IoT also needs messaging, so.... XMPP and IoT!" way. This is the sort of page that clogs search engine results with useless non-content.
If one complains about extensible protocols, one should be willing to come up with some other approach that works.
There's no place for XML in iot, and definitely no place for XMPP. Even json is luxury you often can't afford, not if you want to make a reasonable cost target or battery life.
Why you might want it? Well, maybe for the same reason any home router has a built in web interface (despite all the overheads of having a kinda webserver on such device) - for user's convenience and interoperability.
And yes, stating that JSON is somehow better than XML for such cases is just... wrong.
MQTT is the way to go for IoT.
MQTT at the edge, Kafka at the core.
There is a good Erlang MQTT broker: emqtt / EMQX.
(A feeling I'm very familiar with these days as I work in the telco business - an entire business that knows fuck-all about how you create standards that lend themselves to clear implementation)
It kind of started off on the wrong foot by using XML. XML is too complicated. There is too much freedom, and freedom to screw things up. It also puts too much flexibility and semantics in the wrong place. Complexity, gnarly performance characteristics and design dilemmas abound.
And not paying attention to important detail such as cheap and predictable message framing is such a boneheaded mistake - you have to pay attention to things that make implementation awkward. It is the first problem you need to solve when creating a somewhat performant implementation and you find yourself stumbling out of the gates having to compromise. Sometimes by having more than one XML parser - one to do framing and one to deal with messages.
It has been many years since I worked on anything XMPP-related. I have no idea if XMPP has improved (probably a good idea to spend a weekend playing with XMPP again to have a look). But if it still depends as heavily on XML, it is not for IoT.
Please do not put it in IoT devices. Please, please, please.
IoT spans from devices with systems more powerful than your desktop computer (in kilowatts and MIPS) to small systems that have perhaps hundreds of BYTES of RAM.
While microcontrollers are getting bigger, with some having megabytes of flash and even hundreds of kilobytes of memory, the most interesting growth areas are in devices that use very little power. Ofte as little as a dim LED. And when they sleep, which many do most of the time, they need to draw so little current you'll spend many sleepless nights poring over datasheets to figure out if there's a voltage regulator out there that doesn't blow your entire energy budget.
For the big systems you can do whatever you want. Go crazy. For constrained devices, not so much.
If you need some infrastructure for communication that is to be useful, you have to be able to scale down. Paradoxically scaling up is easy - scaling things down is hard. It tends to make systems grow ugly warts when you create adapters all over the place to translate between domains.
And it isn't just about size. It is about asymmetries in communication patterns as well.
More primitive protocols are quite popular (like MQTT and CoAP). Things that can be done with more or less constant memory and CPU requirements. But they're not necessarily great for everything. And people do not necessarily understand how to design with them. They tend to exist at the edges of systems and concern themselves with hauling data back and forth between constrained devices and the luxury world of more powerful computers.
Then there's the traditional embedded world where it feels like you are always dealing with hermits that got cut off from the rest of the world 25 years ago. Things are getting better as the embedded world is gradually being invaded by more mainstream programmers, but it is painful to watch. People who aren't very good at software versus people who aren't very good with constraints.
What is almost shocking is when you get a peek inside the kimono of different embedded companies / chip manufacturers and you realize the stark difference between where they are in a developmental sense. Some understand where they need to go, and mainly struggle with getting there - others just have no clue how far behind they are from where they need to be.
And then mix in politics, companies that are hysterically protective about their IPR, and the time pressures that exist, and you have multiple culture collisions at once.
I can understand the attraction of XMPP. I really can. But if you ask me we need to do messaging over again. And it has to do less - not more.
There's a circle numbered "9" in that same bad place, for browser developers who refuse to allow end-users to easily stop keystroke-hijacking and scrolljacking with a simple preference checkbox.
Explaining what I'd like to do to either group would probably land me in circle #5, and carrying it out would land me in #7.