Crossbar.io – an open source platform for distributed and microservice apps
crossbar.io
crossbar.io
- Pleasant to use and does almost exactly what it advertises
- Interesting authentication & permission patterns, unfortunately we couldn't extend it to the frontend, and that left us with a strange gap
- Hard to scale for serious cases, clustering is impossible in the open source edition and the commercial version costs in the 5 figures per cluster node
Ultimately I am quite confused about the target demographic of Crossbar (factories moving to IoT?)
I don't know of many pubsub + rpc solutions targeting a similar tradeoff with a tight packaging and a similar learning curve. Here are some I found then:
https://deepstream.io is the closest and has a richer feature set (including a replicating document store), however it seems to have a small ecosystem / community / limited support options, and permissions do not seem as advanced.
https://nats.io is close but lower level. Has a large community and rich ecosystem however.
Then you can also mix and match lower level technologies to achieve the particular set of tradeoffs you need. MQTT, AMQP, WS as the protocol, mosquitto / rabbitmq / zeromq, then json-rpc on top, grpc ...
I'm really tired of these fake open source projects. If you really believe in software freedom, you simply don't build or distribute proprietary software.
Open core is bullshit. It's like a free software kernel distributed with a closed source libc. Nobody would call that project open source.
seriously.
There is no moral imperative associated with open source software; its goal is to leverage the community for more effective software development to further business goals.
Free software is all about sharing as a moral good. But as far as I can see, Crossbar doesn’t claim to be part of the free software movement, only that it is open source.
Just wanted to pitch in with some of deepstream features.
Deepstream currently has its own JSON/protobuf protocol that is similar to WAMP but also has presence and data-sync built in. The downside is that to get the full feature set you need Deepstream SDKs (which didn’t work out since they take too long to create and our core C SDK attempt for multiple languages failed miserably). So currently it’s just a very mature JS/node library.
Permissions and authentication are also built in. Every single message that goes through the system (subscriptions and payloads) can be permissioned either through custom code or using a custom library called valve, which allows you to provide a config based permission file. You get access to the topic, payload, action, user session, and can even do lookups on the cache for data sync cross referencing.
I spent a while after the company behind it (deepstreamHub) shutdown and released V5 under the MIT license which was an almost complete rewrite in typescript that allows you to:
- add an adaptor layer for any real-time protocol that allows multiple protocols to talk to each other. So in our case was MQTT / JSON / Webpack / HTTP all supported on the same server chatting to each other.
- a complete redesign of meta event listening. Which means any client can get notified when a user subscribes and unsubscribes to a topic (whether an event or a record (data sync object), as long as they have permissions to
- a completely customisable plug-in architecture that allows you to hook into internal events.
- a monitoring plugin that allows all message counts, errors, invalid permissions and general stats to be accessed via a http API
- a fully working cluster that allows each node to be aware of the subscriptions on other nodes for smart data distribution (rather than a very noisy message bus)
- real-time filtering on both rethinkdb and mongodb as a separate process. This allows you to automatically filter tables based on push notifications
And a bunch of other features (multiple database, caches, message bus and other plugins).
Unfortunately the tech itself never really picked off, I think the custom protocol was a large downside since it required custom SDKs and the VC backed nature doesn’t go hand in hand with OS, as mentioned in other comments.
I stepped down as a maintainer over the last year as I found it hard to justify the cost of maintaining such a large feature set with such low adoption. It’s hard to say goodbye to a project that you spent so much time on, and I sometimes wonder if a rebrand and new documentation is all it needs. Adding wamp support can be done in a couple of days, but I don’t think lack of features is the issue behind its lack of success.
Shout out to Wolfram Hempel, the original author of deepstream, and Caplin (a UK company that provides a real-time server called Liberator that powers a few large banks trading UIs) where the inspiration for deepstream was heavily based upon.
Anyways, thanks for reading this far! If you came by deepstream before and didn’t use it for any reason, would love to hear why so I can improve any future project attempts.
- The clients are huge. I should preface I'm a Software Engineer that focuses on the web, so this matters a lot to me. They really need to be run in a web worker if used, (they're 1MB + of JS, and don't tree shake well. They don't export ES Modules either, last I checked). We have real time chat infrastructure for context. Web Socket Support in Workers Can be weird too
- There's no fall back to a different protocol in a practical sense (yes, the specification does not state it has to be websockets, it just defaults to web sockets). Its for all practical purposes, web sockets or bust. Why this matters to me? Because Server Side Events would be a more efficient way to push messages to clients when the total volume of pushes is several factors greater than pulls, from the server side. That is, the server in aggregate is going to push more data to all the clients than any one client is going to push to the server. Therefore, it would have been nice for this to be abstracted around different models for flexibility of architectural approaches.
Ultimately we did go with an SSE push (from the server) model, and just send individual requests via HTTP Requests. It actually was less load on the server and faster over all (updates were still extremely fast, but no one client was ever pushing to the server more than the server was pushing out to them)
That sounds interesting! I've implemented mqtt in a chat project but have not investigated SSE as a fallback/hybrid approach. Are you only using SSE or hybrid? I guess I need to read more about SSE but I am curious to compare it to websockets/mqtt and learn tradeoffs.
Its really...simple. SSE streams are unidirectional so you're only listening for incoming messages, never outgoing on the client, conversely, you just broadcast out to your clients over SSE. Just have an HTTP/2 connection makes it even more efficient.
Its not great if you need users to have multi-tab workflows (you can only have 6 connections over HTTP 1.1 for a single user), but that has not been an issue for us. There's some logic you can do on the backend to close duplicate connections.
When Web Transport hits, I'm hoping we can do SSE on the client as well, and have two separate unidirectional (one for the server to push to the client, one for the client to push to the server) and have SSE both ways, this way we can terminate the streams independently for efficency
And to answer your question, crossbar is an implementation of WAMP (Web Application Messaging Protocol) that is built on top of web sockets. If you are familiar with 1990s technology, think TIBCO for the web.
We used it at a startup in the mid 2010s, and it was great for real time communication between clients (browsers) and servers. Never had any issues.
+10000
> If you are familiar with 1990s technology
Is this extremely-subtle trolling?
From a developer perspective, you can create a "channel" and then start transmitting messages on that channel. Then someone else can subscribe and can listen to that channel.
In the same way, someone can "expose" a single method. And you as a developer can call that method from anywhere.
The medium of communication is WebSockets. The protocols it uses is WAMP.
Which means any programming language can implement this (on top of WebSockets) and can take advantage of "distributed application architecture".
One problem was whether it would support horizontal scaling if you needed to expand beyond a single node, and at the time they were "working" on a global/cross router but it wasn't ready. It's a hard problem to solve and one that may not be required for many use cases. I'm not sure if they delivered it but I ended up going with mqtt instead.
To piggyback on another commenter, their javascript library did feel a bit bloated/dated, but I also got the impression a few of the people were juggling a lot of related projects and maybe there wasn't enough of a community. Haven't checked back recently.
In any case, the project is still impressive, has some dedicated people behind it, and could be a great solution to add websockets to a project.
I'm looking at that "Powering the Internet-of-Things" animation on their main page, and immediately asking myself "what if there's a network outage just as the temperature rises?" "what if the fan controller has a power hiccup and fails to start?" etc etc.
Because those days it takes days (or even weeks) to subscribe to the idea, build a sketch and start testing failure scenarios.
Definitely suffers from some of the challenges mentioned above (commercial clustering, large-ish frontend libraries, etc) but it's been pretty straight forward to work with and scales up well enough for our needs (a few hundred to a few thousand simultaneous concurrent players).
The Broker may allow for configuration on a per-topic basis.
--------
Slightly confused, my impression is that it's At Least Once with the client responsible for dedupe, but it says "if you want exactly once", so I'm not sure if the client is supposed to make a second request for confirmation?
https://wamp-proto.org/_static/gen/wamp_latest_ietf.html#fea...
build on top of WAMP: Web Application Messaging Protocol
https://wamp-proto.org/comparison.html
Another alternative that I follow with great interest:
https://github.com/nanomsg/nng
pubsub is a powerful pattern