Conduit: Simple, fast and reliable chat server powered by Matrix
conduit.rs
conduit.rs
That said, it’s been really good for me. Reliable chat between my and a few friends, plus some big-ish rooms that I participate in.
The author also works on Veloren[1]—another fun Rust project!
[1]: https://veloren.net
That game is fucking cool. A couple years ago me and some friends compiled it for kicks and giggles to see what it was all about. To our surprise, the game ran silky smooth on my GTX 1050 Ti. We get into a lobby. We play for a half hour.
We're just exploring (the map is barebones but has cool landmarks) when I realize that you can scroll to zoom out from your character. In a psychadelic twist, you can just keep zooming out past normal ARPG levels and into a minimap-scale world, then above the clouds, all without dropping a frame.
I don't know how well-maintained the project is today, but it's got the bones for a badass RPG. One of these days I'll hop back on...
If the last time you played was a few years ago, they've pushed out a huge amount of features since then. Very impressive for a fully open source game.
"Veloren is a multiplayer voxel RPG written in Rust. It is inspired by games such as Cube World, Legend of Zelda: Breath of the Wild, Dwarf Fortress and Minecraft."
I love how pointing out that it's written in Rust is more important that describing what the game is about or inspired by. Classic open source project unsure of who's the intended audience.
Our target audience is just as much potential contributors as it is actual players. Mentioning Rust is very deliberate: it pulls in a lot of motivated developers, both those trying to learn the language and those with expert knowledge looking for a creative, relaxed way to put their skills into practice.
Updates every Thursday! If you ever get on the main server, look for "varsderk" and I'll help you bootstrap fast. :)
I'm the only user on my server and I've not joined any very active room, but so far its impact on my small VPS performance has been negligible.
Sounds like AppService/bridge support is on par with Synapse at this point (and maybe has been for quite a while?) - any gaps to be aware of in that department?
You can find some bridge documentation here: https://gitlab.com/famedly/conduit/-/blob/next/APPSERVICES.m...
Any thoughts on matrix-p2p aka pinecone which is now being developed against dendrite? Any thoughts on "p2p-ifying" Conduit in the foreseeable future?
https://github.com/matrix-org/pinecone
https://archive.fosdem.org/2022/schedule/event/matrix_p2p_pi...
I've been using Conduit for 10 months now and I love it. Thank you so much for it!
I've two questions:
- Is there any concern to have with regards to its future when you finish university? You seem to be by far the most active contributor and I'm worried the project is still dependent on how much time you can afford to put into it;
- What is the best way for a Rust / Linux developer to do a first impactful contribution to Conduit? With 155 open issues on GitLab at the moment and no problem really standing out for me as a user, I don't know where to start :p
Thanks!
BTW I hope you land a great job; I'd happily recommend you where I work, but we don't have any office near Dortmund unfortunately… Feel free to reach out to me if Dortmund / remote is not a requirement.
1: I can't say how much time I will have for Conduit, but I think the project is in a good shape and I think it can reach a stable release without me working full time on it, but of course it will take longer.
2: I think a good way to start is to hang around in the Conduit Matrix room and see if any issues pop up. Often these are relatively simple things like "these logs should have more details" and are a good way to get started.
I will also use this opportunity to link my LinkedIn profile: https://www.linkedin.com/in/timokoesters/
1: That's a relief :)
2: I've just joined `#conduit:fachschaften.org`; we'll see if I can help somehow.
I'm also prioritizing features needed for smaller instances, instead of things like user management or horizontal scaling.
Thank you and kudos for all of your hard work on this. I will definitely be checking your project out. :)
-sydbarrett74
I think most other things in the spec are necessary complexity. It's annoying to work on logic for threads, spaces, read receipts, read receipts in threads and so on, but they allow Matrix to have a lot of great features.
What problems did you encounter writing bots for Matrix?
> It's annoying to work on logic for threads, spaces, read receipts, read receipts in threads and so on
Another impression I got was this was kind of an excessive amount of duplication (and other duplications) that could have been avoided had a bit more forethought been put into the protocol design, especially with how these features interact with the e2ee.
https://matrix-org.github.io/synapse/v1.85/admin_api/media_a...
My server is not federated but fairly active, and we treat it as ephemeral. We've configured it so anything older than a week or two gets reclaimed automatically, text and media.
>I shrunk my homeserver database from 100GB to a little under 8GB, during a long maintenance cleanup.
I run the compressor on a cron job. I've been running for 5 years with ~ a dozen people on my server, all federated, in multiple large rooms, didn't run into space issues until this year. On Linode $10 and then Hetzner 4 CPU.
One thing to watch out for: disable registration. Otherwise spambots will register accounts, join the largest matrix.org rooms to spam, and then you'll have to store all that crap for all those rooms.
Either way, it's unknown how long it'll take.
The tracking issue for Synapse -> Dendrite migration is all I've found. https://github.com/matrix-org/dendrite/issues/1705
Conduit Beta – Matrix chat server - https://news.ycombinator.com/item?id=28385840 - Sept 2021 (69 comments)
Speaking as a user of both element generally and Cheogram/JMP.chat.
I still use both.
With the new protocol-level performance MSCs coming around the corner, Matrix is becoming ever more appealing for all sorts of applications.
- modern chat solution with features people have come to expect from Slack and Discord
- many bridges that have kept me from having to use more than one chat client for multiple protocols (think pidgin)
- lots of great communities are on matrix
- pretty good voice chat
- open source
- selfhosted
Those are my main reasons why I like Matrix. My question to you would be: what do you like about XMPP?
- plenty of fast native clients
- server resource usage
- simplicity of the protocol, especially the encryption
And God help you making all of them talk to each other properly...
only simple for developers though - due to the lack of Cross Signing it is very difficult to keep communication secure.
Without it, e.g. if you log into a new session, you have to verify with all your contacts before you can safely use that session for communication
XMPP gained bad publicity as overcomplicated XML based protocol which was abandoned by Google in favour of their proprietary solution somethere in 2011. Till then it had been worshipped as one of the best open standard ever developed.
Source: I use XMPP and Matrix daily.
Something with Gajim's features but using Qt would be pretty great potentially. GTK stuff also often has this horrible thing where it fades the window when unfocused, which gets really awful and unresponsive when things get laggy, also ruins screenshots. Never figured out how to turn it off. Part of the theme apparently, but GNOME also doesn't want you customizing/changing your theme. I'm just using Adwaita-dark usually.
Matrix clients aren't great either, but overall Nheko gives me less trouble and pain than Gajim, using both daily. irssi (IRC client) is kind of my gold standard for stability/performance/features with weechat being mostly okay too.
I was there when they had the famous video-voice branch.
Heck, I just checked: https://developer.pidgin.im/wiki/vvAPI?version=1
It was 15 (!) years ago :-)
iOS story is sad. Family used Snikket (thanks for your work on it) but it still dropped notifications even after following with issues and Prosody modules needed. (Monal seems to be okayish now).
Even Conversations.im seems stuck. I know everyone has their favorite missing feature but for me no reactions is the biggest.
> My question to you would be: what do you like about XMPP?
Same things & more:
- modern chat solution with features people have com to expect from WhatsApp
- many bridges that have kept me from having to use more than one chat client for multiple protocols (think pidgin)
- many great messengers have discovered XMPP as a great base for their system
- lots of great communities are on XMPP
- realistic participant counts in public chat rooms - no counting dead (former) members
- pretty good video & voice chat
- open source
- selfhosting possible
- fewer concerns about data protection and privacy
- more resource efficient (less memory required, lower CPU utilization)
And:
- The use is possible and allowed everywhere where e-mail is in use.
- simple encryption support. should be able to set it up with a couple clicks and don't have to worry about it
- seamless synchronization between devices
- quick push notifications that don't require me to have something cluttering up my notification bar forever
these are just some of the points I could think about at the top of my head. the majority of clients have one or a couple of them, but there's no client to my knowledge which has all of them.
I have yet to find one that doesn't have awkward push notifications. like come on, do you really think I want an ugly notification turned on at all times watching for messages?
edit: formatting
Conversations has great support for OpenKeychain on Android, and I'd love to see that come to competing protocol Deltachat but the devs are focusing on cross-platform behaviors atm. Android-specific fork Deltalabs might be willing to be upstream for it if anyone knows where to start with that.
Matrix has a more defined set of important features, so you could expect that conforming clients all implement them uniformly, without surprises.
I imagine there must be some good clients for XMPP. Hell, I use one good client (Conversations on Android) on a daily basis. Also have used Gajim quite a bit. It is alright afaict.
It's not even possible there, because you have conflicting proposals while on Matrix you have a coherent specification
That said, I can't claim XMPP clients are any better.
Classic Element should support every specced feature in Matrix tho, and tonnes of MSCs, so unsure what features you’d be missing. Meanwhile Element X is less featureful, but way more performant and stable (we’re aiming for better-than-telegram UX and perf).
Of course you are already familiar with https://matrix.org/ecosystem/clients/element/. I know it is pretty ticky to point out that "Multi Account" is not supported, but it happened to be a feature I was particularly looking for in my search. My IRC client, for instance, lets me be logged into multiple servers with different accounts at the same time. Yes, I know that a single account on one server can communicate across all federated servers. The same is true for email, yet my email client allows me to use multiple accounts. I'm heartened to see that you have included the feature on your checklist. I hope that means somebody is thinking about it.
I know that Element should allow me to paste into the composer. I was told exactly that in the matrix room when I asked. "Works for me" is as frustrating a response as ever. I've seen the issue reported a number of times by different people with pretty decent detail, but I haven't seen a solution. Just "works for me" responses.
I'm still a Matrix user. I just accept that I don't have everything smoothed out the way I'd like. My post above was just a response to the claim that XMPP clients don't have uniform features. Well, as you know, neither do Matrix clients. But you weren't the one making that argument.
Don't take my posts as negative criticism, even if there is some criticism there. Keep up the great work. Keep getting better.
Oh thank god - my (admittedly not-brand-new) phone doesn't have quite enough RAM to keep Element fed and happy. 300 meg doesn't sound like much till all apps are limited to the ~2GB left over after the OS takes it's cut.
Element supports stickers though, and I haven't found any other Android clients that support stickers while also being lighter.
Kudos though - Matrix has come a long, long way since I first played around with riot.im!
It requires the clipboard API. Puzzling decision, given that native pasting worked before, but here we are.
Matrix does not work this way.
No.
XMPP is just as "modern" as Matrix, but has a _different _ data storage/distribution idea.
XMPP has had little place on the public stage, but that doesn't mean it stopped evolving.
if and for how long will be decided by the person who is running the server the chat room was created on
I also prefer XMPP.
And I'm looking forward to integrations (Slack, Signal, ...) being more user-friendly, and easier to install, configure and maintain, so I can message others, too.
I’m not sure if XMPP works like that, but I always thought it was more like email: send a message, maybe retry a bit, but that’s about it. Not sure how transient things like presence, read receipts, and typing notifications work either, across servers.
XMPP is the only system that can and may(!) be used wherever email is in use.
XMPP is just as "modern" as Matrix, but has a _different _ data storage/distribution idea.
Comparison: https://www.freie-messenger.de/en/systemvergleich/xmpp-matri...
https://news.ycombinator.com/item?id=8998290
> XMPP is great for what it was designed for. It doesn't work well with mobile, high packet loss & high latency connections. XMPP is talkative and bandwidth intensive - bad for limited data/battery applications. It also wasn't designed for today's 1 person multiple devices reality. Most XMPP servers let you log in multiple times but messages don't sync between clients and sometimes get delivered to the client the user isnt currently in front of.
> Also, sending files over XMPP has pretty much always sucked - there are a bunch of incompatible ways to do it and it's always been hit and miss depending on which client your chat partner was using, network topography, etc.
Also, overload of XEPs doesn't help the ecosystem. Too many optional extensions hinders interop. People expect more features in modern chat experiences than what XMPP was designed for, and that's what XEPs have tried to fix as a bandaid, with mixed results.
On top of that, the technology choices are par of the course for the time it was designed, and nowadays there are arguably better things. Devs are naturally driven to choose tech that makes their work nicer, if they enjoy it, so that means more stuff gets done for the newer platform, in this case. As an example, coincidentally, another HN post today was touching on one of those points - the need for a very advanced XML parser, as a typical one apparently wouldn't be enough:
This is an absolutely false, baseless statement, mostly amounting to FUD. Messages work well on mobile, and they sync between devices just fine.
> Also, overload of XEPs doesn't help the ecosystem.
On the contrary, XEPs create the ecosystem. As I often say, Matrix is not a protocol, it's a product with an API created by a single organization.
I'm excited for the recent progress on the built-in Sliding Sync feature because it's much easier to deploy.
Not all messages are encrypted with the same key, so if all of your clients are not connected at the same time, and the same is true for the sender, they can't exchange their keys. When that happens, each client can only decrypt the subset of the messages for which it has the keys. Also note that clients only exchange their keys with other verified clients.
If you look at the “session_id” attribute of the JSON source of the messages, you'll see that for a given session (ie. when the sender is logged in a client), all the messages are decrypted (which means you have the key for that session) or none of them are (which means you haven't received the key for that session yet).
What is Matrix?
Matrix is an open network for secure and decentralized communication. Users from every Matrix homeserver can chat with users from all other Matrix servers. You can even use bridges to communicate with users outside of Matrix, like a community on Discord.
And I see no installation information.
Installation is downloading a binary, setting up autostart and writing/copying a 10 line configuration.toml.
You probably would want element.
I think conduit's page is doing the right thing. It doesn't really assume you know what matrix is, but rather assumes you're smart enough to click on a link if you don't know.
> Users from every Matrix homeserver can chat with users from all other Matrix servers.
This statement lead me to assumed the "matrix specification" discussed was about server-to-server communication... I'm not interested in digging into the underpinnings of this server-to-server communication (right now) just like I'm not terribly interested in how blockchains keep in sync across nodes... But I am interested in how this whole stack works, and I was lost with this page as the starting point.
So I think it's safe to add an extra sentence to explain the relationship while still keeping the page simplistic and focused.
EDIT: realizing that the landing page may have been updated with new content itself since your original post.
Matrix is a chat protocol. It works like chatting over GIT (distributed databases) and is more like a team messenger like Mattermost or Zulip. Conduit is a server software and not for end users.
More information about Matrix: https://www.freie-messenger.de/en/matrix
if at all, Element is this.
Matrix doesn't really care about what type of users the clients target
Separately, it seems that Conduit is not orienting itself as a drop in replacement for Synapse. At least at the current moment. There are a number of notable feature gaps that I recall being mentioned last time I was in the Matrix room.
The selling features are lightweight and simple to setup, implying that Synapse is not. It is more importantly a validation of the matrix server specs and over the years they have done quite a bit to get ambiguities clarified.
It's a shame there's no upgrade path though. With retention of history.
At some point Dendrite was slated to support micro services on different machines to be able to scale up, but now they only support monolith deployments. It's aimed at "small" deployments and is used in their P2P efforts.
Possibly, one of your clients has the keys needed to decrypt one of the messages, but you're using another client which doesn't. Things go back to normal when both are connected at the same time and can share the keys, or when the client of the sender is connected and still has the keys.
If you don't keep your clients connected all the time, you can use a secure backup on the server, so the clients can retrieve the encrypted keys from the server and decrypt them locally.
Not having the keys happens more often if one the parties uses short-lived sessions (like logging exclusively in a private browser window, for example).
This article helped me with understanding a little: https://gerstner.it/2021/02/matrix-and-e2e-encryption-or-how...
Specs: The server we use has 4 cores (barely uses 1) and 8 gigs of RAM, 2.5 of which are currently used and 120gb of disk space, that will probably suffice for another 2-3 years.
Its been working without problems for a single user setup (no VoIP though).
I hope you get an idea of the minimum resources needed.
So 100s of users wouldn't be much of an issue if they're mostly in the same room.
However, I also spent a lot of time configuring my Synapse server with tons of bridges, so even though I totally get the need for more "lightweight servers" (especially in order to break the Synapse de facto monopoly on Matrix implementations), I'm unlikely to use it as a primary driver unless some killer feature (like native Fediverse integration, at least via Fedi authentication) is implemented.
https://spec.matrix.org/latest/
Any reason why it's not a good idea to integration the API with server side (aka E2E or distributed drawbacks?)
On the other hand, their dev matrix channel is very active. Props to that. Still...beta software.
I have never managed to self-host a Matrix server for my home because they all demand a domain name like matrix.something.com. I can't use a local address like 192.168.1.100.
If you want a private, non-federated chat server -- for your organization, basically -- Zulip is awesome.
If you want federated chat, Zulip doesn't do that.
you can use a mozilla.com user id to talk to someone with a matrix.org user id, or a kde.org user id
all servers are equal participants in rooms, so a room doesn't live on a specific server aside from servers being able to create friendly names pointing to them, but nothing stops #foo:kde.org from pointing to the exact same room as #bar:mozilla.org -- both servers are participating equally and have their own shortcut name to the room
Being able to leave one server and join another while maintaining an identity (say, a public key for instance) is on Matrix's to do list, they haven't decided how to do it yet afaik.
edit: Okay, not actually webfinger. But [0] has instructions for the `/well-known/matrix/server` way. It only talks about subdomains, but it should work across domains. Possibly also with the SRV header.
[0]: https://github.com/spantaleev/matrix-docker-ansible-deploy/b...
No.
Matrix is like chatting via GIT (distributed databases) and tends to be a great team messenger like Mattermost or Zulip - XMPP is structurally like email ...
And I'd say that Matrix is closer to e-mail and XMPP than you seem to assume. Once the database is synchronized, it works pretty much the same way. Only if a missing message is detected in the graph, then a server makes another request for the messages it missed (backfill).
Moreover, Matrix and Git are quite different, since you want to be pedantic. Both the synchronization protocol, and conflict resolution are handled differently (Git does very little, Matrix is more like a CRDT in that respect).
How can I "transfer identity" to a new domain in email?
I can't move myusername@gmail.com to @outlook.com, nor can I move myusername@someisp.com to myusername@anotherisp.com .
Matrix (Synapse / Dendrite / Conduit / Element) all support audio and video as well as text.
This will always be an "issue" of decentralized services, just look at the Lemmy devs over on Lemmygrad, super far left. Anyone can host a server running a protocol, lets not throw the baby out with the bath water just because people are saying something objectionable in one server. If you don't like these people join/make an instance that doesn't interact with them.
I’ve found many nice and welcoming communities on matrix.
The only time I've encountered anything of this description is when I was looking into why a server had been defederated, and then found out it was mostly neo nazis and weirdos obsessed with anime children. I had to actively seek them out to discover this.