Matrix 2.0: The Future of Matrix
matrix.org
matrix.org
I am excited to try out Element X, but I do not want to administer yet another service, namely the Sliding Sync proxy. From a practical perspective, Sliding Sync (MSC3575) is still on the level of a proposal/experiment and I wish that we would be having this conversation about Element X after it becomes usable with any spec-compliant server.
Sliding Sync is an interesting case in that it clearly does work, and it kicks ass - but it turns out to be unnecessarily complicated for the subset of features that you actually need when implementing something like Element X (which is my fault entirely, fwiw). However, should we hold off on putting it in people's hands while we sort that? Nope. Is it a pain to temporarily run a proxy shim service while the protocol stabilises? Yes. It's like running a SPDY or QUIC-aware reverse proxy in front of a webapp that only speaks HTTP/1.1 while waiting for SPDY to be ratified as HTTP/2. Sure, it's another moving part to look after, but the spectacular improvement is likely worth the pain. YMMV though :)
Yes, it is just unfortunate that it is not clear enough to more people (such as the_common_man below) that Sliding Sync is only at the MSC stage at the moment and not part of the Matrix spec.
I am glad that at least Element X allows this MSC to improve and for those nits you have mentioned to be resolved.
Once the API is stable I'm sure all the homeservers will implement it natively. Until then, think of it like a reverse-proxy that lets an HTTP/1 webapp talk HTTP/2 to the outside world.
I would then ask why it isn’t integrated into Dendrite
Of course.
My argument isn't that it won't happen, it's exactly the opposite, that it will. My point is that, now you need to manage 2 things vs 1.
> In fact, some people might actually prefer the fact they can operate it separately and restart/maintain it separately from their homeserver :)
Of course. But it's misleading to say that it won't require maintenance (which is what I replied to).
1. Added a new postgres db and user 2. Added a new dns name 3. Added a new cert for the new dns name 4. Added a new vhost to nginx to proxy to a new port using the new dns name and cert 5. Added a simple docker-compose to my config 6. Added a snippet to my .well-known/matrix/client
Don't imagine it will take much maintenance. Container will be updated along with the rest of my containers, and the existence of the service doesn't change anything for any existing clients that don't support sliding sync.
The fact that this is independent of any server implementation (read Synapse) means you can release much more frequent.
Choosing to use a proxy here was the right decision.
https://github.com/spantaleev/matrix-docker-ansible-deploy/b...
I'm really tired of the litany of buzzwords and excuses that the Matrix team comes up to cover-up this large sunk cost fallacy of a protocol. Nobody needs a distributed graph database and state resolution to prop-up a chat system. It has not proven to have any real-world benefit nor served to build a niche in which Matrix is in any way better than the competition. It's been a failed experiment for about 10 years with no end in sight. At what point does the community wake up to all the nonsense?
Perhaps the problem they're trying to solve warrants that complexity, but my particular use case doesn't. I just run XMPP now.
Its flat wrong to say that it can't be efficiently implemented on the client side, the server side of the Api is the problem, not the client.
UX-wise, it made me appreciate the distinction between Discord's "everyone is subscribed to every channel in a server by default" stance, and IRC/Matrix's opposite approach. The latter certainly scales better, but the former is so, so much better for discoverability, so it's probably the better default for small servers.
And if you're looking for a Matrix client with a Discord-like UI, try Cinny.
* The ability to use the end-to-end encryption identity & verification from your normal Matrix account, by piggybacking on the 'host' matrix client. Whereas call.element.io is not remotely designed (at all) to work with your normal Matrix account, and doesn't have any of the E2EE UI hooked up, and instead relies on a shared secret to access E2EE rooms
* The ability to place VoIP calls inside existing Matrix rooms from your normal Matrix account.
* The ability to reliably background the app & lock the phone when on a call.
* The ability to receive inbound calls via push notifications and CallKit/ConnectionService - i.e. presenting the calls in the native dialer UI of the phone, and treating Element X like a VoIP dialer.
You get the same beautiful UI whether you use call.element.io directly or embed it inside another Matrix client like Element X :P
That said, one thing that did help a lot was having the server host a default client. Anyone can still connect to the server using any client they want, but now they can also go to the server's URL and get a good client without any hassle. We chose Cinny as the default client because it's solid and extremely familiar to any Discord users.
Not sure why this feature keeps getting defered for "just about to land and things might be better after" server-side features (see how it has 7 out of 9 "dependencies" resolved) when none are needed.
I've had a couple of friend actually give up on matrix due to this. I also have to keep my phone on silent or else I can't be chatting on matrix without the constant bzzzing.
I'm also amused about the ever growing (currently at $760) bounty on this feature.
I and several others have also given up and just stay with Signal and WhatsApp.
A project is always in large a reflection of the leaders. It’s the messy hackathon type project - not the clear and polished corporate project.
Meanwhile, we massively fixed Element Mobile's UI a year ago: https://element.io/blog/an-unrecognisable-improvement-elemen... - and Element X is essentially the same UI... but on a different engine. So I'd argue that we were very aware that Element's UI was crap, which is why we fixed it. Rewriting the Element mobile apps as Element X is unquestionably the right thing to do in terms of sharing the same rust codebase between the two platforms and letting the app layer focus on UI.
Agreed that our progress has been slow, and our focus been has very broad at points. But we got there in the end, and certainly have learned from the journey (and built Matrix to be a much broader and successful heterogenous ecosystem as a result).
I'm sure your projects are all much more successful though - I'm jealous! :D
Having had a look at Element X (Android), it actually feels to me like that responsiveness is pretty much the last thing I was missing.
Oh, and the call bug already mentioned in the thread.
I'm really happy with Element and thankful for all the work you guys put into it.
I use Element after switching from Google Hangouts/Chat/$LATEST_NAME$ a long time ago.
What, specifically, are you saying has always been crap about it?
Personally, I have no complaints about Element's UI. It works great IMO.
And I consider myself pretty hard to please and easy to irritate.
Then it would be more obvious who said what.
But X requires Sliding Sync on the server, which is still a separate service to run alongside the homeserver and doesn't have a stable API. I am increasingly disappointed with how centralized Matrix is becoming, since AFAIK there isn't really an alternate client close to the same level of quality as Element.
I probably would've made all of the same decisions myself though, so I don't blame them I'm just a bit disappointed in how it's shaking out.
So it might not be that easy (if even slack with their funding can't get it right).
A bit of a privacy leak on shared networks revealing which IP is doing messaging, but it should be fast and reliable.
A pretty basic functionality which even Messenger supports....
- Adding Rust matrix-sdk-crypto to matrix-js-sdk, at which point all the official Matrix.org client SDKs will (at last!) be using the same stable performant E2EE implementation
- Continuing to contribute Matrix input to the MIMI working group in IETF for > Digital Markets Act interoperability
- Working on MLS for next-generation E2EE
IIRC, the rust matrix-sdk-crypto was their intended fix for the vulnerabilities.
Then, we can address the harder server-controlled group membership issue in one place. First step will be to improve device verification & trust so that trust is the default, not the exception, to make it easier to spot and warn about unexpected devices in the room. The full solution is then either MSC3917 (https://github.com/matrix-org/matrix-spec-proposals/blob/fay...) - or potentially to switch everything to MLS.
We're working on MLS anyway in parallel to RHUL mitigation work; you can see the progress at https://arewemlsyet.com, and it's looking good.
I'm guessing you're not interested in doing a podcast on "yay we converged our crypto implementations on a single robust Rust implementation so we can fix the remaining bugs in one place", but as soon as the server-controlled group membership thing is solved we'll be in touch. Work has also gone much slower than hoped on this, thanks to the joys of funding open source.
There is a huge discussion about that on git and some can't reproduce, some can't get a single notification even though the apps notification checkmarks were all green.
So it had a very very low WAF [1] and I scrapped it sadly. This was a few years ago, any signs of these problems still existing?
I am not suggesting that was your issue but it might help somebody.
Element X has totally rewritten notification code (given it’s moved from Kotlin to Rust) and should be much more robust for bugs like this.
Sorry you got bitten by it.
It works fine, but I'll probably switch back to Matrix's push server at some point.
Edit: With Synapse + Element, obviously.
The copy of Signal installed on my GrapheneOS Pixel (with no Google anything installed) begs to differ; notifications have been both low-latency and rock-solid.
So it absolutely _can_ be done, and done well.
I've had more issues with missing notifications on my OnePlus than I ever had on my Pixel (I have list of reasons why I'll never buy Google hardware again, mostly coming down to their support). Most issues get fixed by me persuading OnePlus to stop applying their secret sauce to power control
It's very unfortunate there isn't a privacy-friendly push notification mechanism built into Android that doesn't ruin battery life. Or iOS for that matter; that might actually be a feature that would get me to switch.
Git doesn’t have discussions??
[0] https://github.com/spantaleev/matrix-docker-ansible-deploy
I really can't wait for Beeper[3] to rebuild their fork on top of Element X (it's currently based on Element, formerly called Riot). When this happens it will be an absolute game-changer for the messaging ecosystem.
————
[1] https://github.com/vector-im/element-x-ios
Great service though and hope if gets UX improvements too, as Discord and Slack keep getting more clutter.
AIUI the protocol was and is called Matrix. There seems to be no intention to depart from that name.
Matrix is a protocol. Synapse is the primary server implementation of Matrix written in Python.
It's worth nothing that the conflict with Riot Games over trademark rights was also a motivating factor.
I bet you were even more bummed though.
It's interesting seeing a myth build around your reasons for the name change already, with the blog post still up. Makes me wonder though how native speakers would have felt about the name after the protests.
I'm not a native speaker, so I don't feel the connotations of the name.
Lock it to the matrix.org server and offer membership in the foundation as an in-app purchase. Could help with some of their recent funding issues.
There aren’t exactly many so it’s a bit hard to generalise but free, large scale chat apps are the side projects of tech companies with existing revenue streams.
Align on a protocol, experiment and compete with clients is a good way to make progress.
Notice how WhatsApp took off while XMPP fragmented.
The Matrix name has never budged since the beginning of the project. One of the Matrix servers is called Synapse. Others include Conduit or Dendrite (see https://matrix.org/ecosystem/servers/).
So yes, Matrix is the protocol, not the/a server.
(To be clear, the platform has to be all open source and open standards, actually decentralized, guided by those principles -- not an effectively proprietary play even if nominally open. For example, you should be able to get all the same functionality with Element, Fluffychat, Thunderbird, and other user agents, and with any home server, both now and on an ongoing basis. And the user agent implementation difficulty should become tractable enough, in terms of size&complexity and specification, that one person could write a new one -- not like Web browsers have become, where there's mainly only one, that company has been funding the runner-up, and that company can pretty much dictate 'standards'.)
Firstly, it's not introducing new features - it's just making the current featureset work super-efficiently (while retaining compatibility with today's Matrix). So there should be no fragmentation between Element/Fluffychat/Thunderbird, with the possible exception of the new scalable encrypted VoIP conferencing (but then very few Matrix clients implemented the old VoIP conferencing, plus you can always just embed an Element Call instance to get at it now - as Element X does today).
Secondly, we're explicitly working to protect Matrix from the commercial interests from Element (the company set up by the team who created Matrix, of which I'm ceo/cto as well as Matrix project lead). We already have The Matrix.org Foundation (https://matrix.org/foundation) as an independent non-profit to hold the Matrix IP and maintain its neutrality - but we also just hired Josh Simmons (former President of the OSI) as its independent Managing Director (https://matrix.org/blog/2023/09/introducing-josh-simmons-man...).
So: going forwards, the Foundation increasingly does not have to depend on Element not being evil, but instead has its own independent management to ensure the protocol exists to support everyone in the ecosystem, without fragmentation, as per the guiding principles at https://matrix.org/foundation - under its own autonomy.
I'd say this is is good news for libre software privacy & security type people, but then again I'm biased. Practically, the only thing getting in the way of getting friends & family on Matrix is that Element X requires fancy OIDC-native homeservers for registration, so newbies won't be able to use Element X to join Matrix yet. But the app itself should be an excellent way to get folks on board in the very near future.
> plus you can always just embed an Element Call instance
That sounds like potentially a commercial embrace&extend&extinguish situation, which we need to avoid.
Admittedly the new livekit-based implementation takes us back down to a single implementation for now, but in its defense it’s only a few weeks old (with e2ee). Hopefully others will spin up independent implementations rapidly - although right now it does introduce a potential commercial EEE situation… but from on LiveKit rather than Element.
Eitherway, it’s something we’re painfully aware of… while also trying to ensure we find enough $ to pay for dev on Matrix (and Element) otherwise the whole thing collapses.
the way threads are displayed in the browser versions makes them hard to follow. in our hackerspace, where many are tech minded people, i see regular complaints about that.
scrollback to unread messages often fails. i have never had another chat application where that was a problem. this should just work.
the handling of encryption keys is very confusing. there are to many moving parts, it's not clear what i must save so that i can restore the keys in a new client.
private rooms are also confusing. i want to have a private room be the name of my chat partner, and they want it to carry my name. being able to do that would be very helpful.
i am also missing the ability to use a different nickname in every room. different groups know me under different nicknames. discord and wechat can do it. it would be great if matrix/element could do that too.
This alone makes Matrix the worse chat platform out there from a users perspective IMO. I invested too much effort in getting things up and running and using it now though, so I'm just hoping that this gets fixed one day, or something better comes along that I can migrate to.
One time I had to manually export and import my own keys to be able to read old messages, which was a task that would have been too technical for most of the people I'd like to chat with, and was supremely annoying for me.
It's just a non-starter. I have a community I've been considering moving to Matrix for literal _years_ now and there are just so many sharp edges.
I've been confused about Matrix's priorities for a long, long time. I am not getting any less confused, to be honest.
The high level plan is:
* Fix all the bugs which cause random undecryptable messages, which was the main thing causing frustration. This is (hopefully) done now by converging on the new Rust implementation.
* Generate a recovery key, rather than confuse the user by asking them to pick two passwords (one for their account, one for their encryption). Impress on the user that if they ever want to recover their history, they'll need it. Expect that most users will lose it.
* When logging in on a new device, scan a QR code on an existing device to transfer over the keys (and thus history). Users are used to this now thanks to WhatsApp and Discord, and empirically they can do it fine. The recovery key is used only as a worst-case scenario if the user has no other devices. The act of logging in (either via scanning another device, or via recovery key) signs that device as owned by you.
* Only accept messages from devices that a given user has signed (i.e. trust on first use for users). If you want to verify that user out of band, they get an green tick too, purely to help you keep track of which users you're sure are legit and which users you've assumed are legit.
* At any given point, encryption is either enabled on your device (i.e. you've signed that device; it can access your encrypted message history; it can store keys for your encrypted message history) or it's not (and you can't do any E2EE). There are no halfway states.
This may sound simple and obvious when written out like this, but each point is a major change from the supremely confusing system we have today. The thing stopping us from having implemented it sooner is the huge amount of effort which went into reimplementing the engine in Rust. But having now (almost) finished that project, next thing up is fixing the UX, at last. The rough timeline is hopefully end-of-year, but, again, timings are contingent on funding pressure.
Having got Element X out the door, my attention at least is going to swing back to Element Web. In terms of encryption disasters, we are about to switch Element Web’s crypto to the same rust implementation as Element X, hopefully next week - you can track the progress at https://github.com/vector-im/element-web/issues/21972#issuec.... Hopefully this will make a much-needed massive improvement on encryption, while also speeding it up 5x or so. On Element X, encryption failures are almost unheard of (other than when talking to Element Web).
This is a centralized point for surveillance of metadata. I also do not know how much information is actually contained in the notification itself. In any case Vector has knowledge of every home server with an iOS client, and their approximate traffic levels.
It also requires that you as a homeserver sysadmin a) have a Mac, b) install Xcode, c) know how to modify and compile iOS applications, d) want to deal with rebuilding and reuploading and re-seeking app approval on major API changes because long-un-updated apps are not allowed in the App Store, etc...
If you start by assuming that a perfect solution will always be unattainable, deciding on what tradeoffs are acceptable becomes easier.
Imho it almost always going to be worth trading the large PITA of self-managing APNS infra for the small surface area that routing push notifications through Elements systems.
The problem is iOS's privacy-hostile design (in service of their locked down revenue model), not some unbreakable iron triangle.
On Android, messaging apps can run in the background and periodically talk to their homeserver without an intermediary.
Background tasks on Android can pretty much be killed at any time. I believe only apps with a persistent notification and the native google notification service can remain active at all times. But yes, they can connect to any server while they are running.
Once you're going to push encrypted data through a centralized service, you might as well just run Signal and skip running the homeserver too.
does this mean the Element X is more for the mainstream user and regular Element is better suited for power users?
The model that we're going for is similar to iOS or macOS: something which is incredibly simple for normal users, but can also scale up seamlessly for crazy powerusers.
In terms of specific poweruser features: I suspect the strategy will be to see if we can provide the same end result without actually adding the feature - and failing that, add the feature very strategically via progressive disclosure... or perhaps not at all, and rely on other Matrix apps to pick it up.
The intention is to fairly rapidly replace regular Element on mobile with Element X, eitherway.
And thank you for your the hard work!
Telegram has been adding a ton of features during the last 5 years, all while keeping the core experience pretty much the same (fast and seamless). Whatsapp is also doing this.
With some UX Design chops, you should be able to do it.
(disclaimer: I have no knowledge about neither spec, I'm not suggesting Jabber was better)
Matrix rooms are fully decentralized and can survive any servers failing.
(Although note that not much else in the protocol is decentralized, much of it is only federated.)
Jaber had something like that 5 years before Matrix: https://xmpp.org/extensions/xep-0289.html but decided it wasn't an avenue worth pursuing, as a non-issue in practice. This is further proven by WhatsApp (which is based on XMPP/ejabberd) and its billion active users: you don't have to shove this complexity into the protocol level when it runs natively on a distributed/federated infrastructure (the erlang VM in this case).
> Matrix rooms are fully decentralized and can survive any servers failing.
In practice the room survives in a "split brain" mode where different users see different participants, the room isn't available to the users whose instance went dark as its no longer hosting their identity, and messages are popping mid history while the state resolution is working hard to reach consensus. When the servers finally come back online, there is a risk of never catching up fully, or worse, for the rooms to remain split forever.
And I adventure a hot take there: this property of staying seemingly online (in a frankenbizarre way) came handy way fewer times in the history of Matrix than for all the incurred complexity reflected in daily bugs, privacy, security, performance issues and inconsistencies.
To add some colour - there are commercial implementations of that (e.g. https://www.isode.com/whitepapers/federated-muc.html ). But from my perspective as a server developer, I've seen minimal interest in this for the public network. Probably because server failures are rare enough events in the general case.
That's not to say there aren't use cases where it is more valuable to have this functionality.
But I don't want to trust my community to someone providing me infrastructure. Lots of projects, communities or whatever start small on third-party hosting. Many open source projects right now have rooms on gitter.im or matrix.org. I don't think they made a "bad" choice and I don't think they should need to scramble to find new rooms if their provider goes offline.
> In practice the room survives in a "split brain" mode where different users see different participants
Even if this is true (I haven't see it when servers go offline) this sounds like a bug that can be improved over time. Not a reason to give up.
But also do you have any evidence of this? Matrix rooms don't have a "home server" so if this was a bug that occurred it would occur for any member of a chat going temporarily offline, and for large chats that happens often. So it seems like if this is an issue it only happens in very niche circumstances.
> isn't available to the users whose instance went dark as its no longer hosting their identity
This is true, as I said users are federated and suffer if their homeserver goes down. But it is a huge difference to be able to set up a second account on a second server and continue participating rather than needing to convince a large group of people to switch to a different room.
Look at freenode for example. It was fairly organized as far as shifting a massive group of communities goes but still quite messy. In the matrix scenario rooms could have moved by simply adding new admin accounts on different homeservers and kicking or demoting all admin accounts on freenode. The users wouldn't have to do anything (except moving away from freenode accounts if they prefer).
> But I don't want to trust my community to someone providing me infrastructure.
I mean, that's always the case in the end, isn't it? Your identity is tied to your server, and that's pretty much what defines the whole thing as being a federation and not P2P (that applies equally for XMPP as for Matrix). As long as migrating from provider to provider is easy and convenient (which amounts to acknowledging that all providers eventually come and go, what centralized service users are in denial about), I don't see it as being a fundamental issue.
It's also easier to mitigate this aspect in XMPP-world where being your own provider is orders of magnitude easier and cheaper compared to running a matrix homeserver and federating at scale.
Also, talking about infrastructure, the situation is quite worrisome in the case of Matrix, I'd say: only one entity is competent enough (or has enough cash committed) to running large-scale: new vector. And it runs that of matrix.org, kde.org, fedora.org, mozilla.org, …. It also happens to be the one specifying and implementing the (incompletely documented) protocol, conveniently.
>> In practice the room survives in a "split brain" mode where different users see different participants
> Even if this is true (I haven't see it when servers go offline) this sounds like a bug that can be improved over time. Not a reason to give up.
It's a very very hard problem to crack. They've been at it for 10 years across many iterations of the protocol: https://telegra.ph/why-not-matrix-08-07 . I like to think that it can be solved the same way I want to have closure about P=NP, but I also challenge that this is a meaningful and practical approach in this context.
> This is true, as I said users are federated and suffer if their homeserver goes down. But it is a huge difference to be able to set up a second account on a second server and continue participating rather than needing to convince a large group of people to switch to a different room.
Yup, and that's all pretty and neat in theory. In practice it happens every so often that rooms just break for inexplicable reasons, and that people must migrate anyway. Not all members do, because for some, everything still looks like business as usual (happened again few months ago with the 3D Printing space), and trust me, this is much more eventful and disruptive than freenode committing seppuku once every quarter century, or with large XMPP networks going under.
In Matrix, your chatrooms synchronise chat history between servers (rather than passing messages around), and that history is replicated equally over the participating servers, a bit like Git.
In XMPP, you pass messages between clients on different servers using a given server as a hub, and clients can choose to back up their message history on their local server.
Which architecture you prefer probably depends on whether you think instant messaging should be about passing messages or archiving chat history.
XMPP had FMUC (Federated Multi-User Chats) about 5 years before Matrix was even invented. It didn't see broader adoption in the XMPP world because the community figured that this isn't a problem worth solving, and/or that the incurred complexity isn't worth it.
And worse, Matrix even hasn't found an adequate and scalable solution for this "proud" ambition (and probably never will). I don't expect that you could demonstrate in good faith that distributed state and its resolution are a net positive for Matrix, its community, its ecosystem, nor that it has opened new possibilities for the practical cases and users. We are back from the blockchain hype/hangover already, and I can't wait for the same thing to happen to Matrix: this was the wrong tool for the job. Just let it go.
And the similarities don't end here. XMPP doesn't have built-in support for encryption (apart from the basic TLS encryption for the transport layer), it doesn't have support for message archiving and chat history syncing, there is no support for group chats, and so on.
A lot of this functionality is added as extensions (e.g. group chats are XEP-045), but this simply caused a lot of fragmentation in the ecosystem. So you could never rely on your client (or server) interoperating with other clients properly.
Audio calls and video also never really worked well. Google tried to push them by releasing libjingle in 2006 (!!!) but it was kinda ignored by everyone.
On the other hand, XMPP desktop clients certainly work better and faster than Element, although some of them look quite old, which doesn't take away from their functionality.
IMO it's being heavily overstated how Matrix is better than XMPP.
Also a nice read: https://telegra.ph/why-not-matrix-08-07
And the answer https://lobste.rs/s/wvi9xw/why_not_matrix#c_erbsnb
Doubly so when you add federation.
It required more than one iteration of Matrix to get it right, which I (personally) totally expected to happen. I don't think XMPP will _ever_ get it right.
Regarding clients, I don't think Element is worse than most (all?) XMPP clients when you look at the functionality past simple 1-to-1 messaging. And forget about mobile apps, XMPP just sucks on mobile.
I would be inclined to call this FUD. XMPP had E2EE before Signal and Matrix even were released. Current XMPP uses a flavour of double ratchet encryption (OMEMO) that puts it in par with those two, while working on MLS (just like Matrix).
> And forget about mobile apps, XMPP just sucks on mobile.
And yet, Conversations uses a fraction of the energy and resources Element requires. And does audio/video calls more reliably. And works well for 1:1 to large chat groups with thousands of participants.
It is a typical situation to observe "Failed to decrypt message" across the whole chat history. It is typical to observe reordering or even lack of messages, which make reading the chat unbearable. It is typical to have problems with session validation.
On one account I have even lost the validated status of another member's sessions, even though we have validated each other before and the only thing that changed is that they logged in with a different client. Since we live in different countries far away from each other, we just have to look at the red shield every time we open the chat and keep in mind that it's not a MITM but just a Matrix failure.
> XMPP just sucks on mobile
So does Element. I have <10 chats joined, and every time I open the Element app I have to wait for 10+ secs before I can start reading new messages. This will surely be fixed by sliding sync, but it's a problem of its own.
Good work.
On Android, I can almost guarantee your bug will be fixed: Element X Android doesn’t share any code with Element Android at all, as far as I know (given the former is mainly written in Rust, and the latter is all Kotlin)
And thanks I'll try that out!! I really hope it fixes that sync bug. Thanks for all your work on it. It's a great platform.
The reason for Element -> Element X is in order to share precisely the same Rust SDK between iOS and Android (and maybe other platforms in future), so all the heavylifting happens in one single audited codebase, rather than doubling up all the development between iOS and Android, doubling the bug count, etc. It’s also a good opportunity to simplify the data flows, which are surprisingly subtle and performant in a messaging app; I’m not sure how much Rx remains. In this instance, the maintaining team remained the same.
Getting a let's encrypt cert is very difficult because my server can't get reached from the public internet. I have it that way because I use this one only for bridging and I'd like to keep it safe.
And before someone suggests it, I tried Revolt, and it's not there yet. The latest client (as of a few months ago) couldn't find my audio devices at all, and came with a big warning about how their audio backend was being completely rewritten, and what I was using was now legacy.
How does ease up sign up and use compare to discord?
Assume that I make the choice of which client to direct them to.
Matrix 2.0 is all about demonstrating that a really really fast, mainstream-grade app can be built on Matrix, in the form of Element X: https://element.io/blog/element-x-ignition/. However, right now Element X is intended for existing Matrix users - so unless you have a bleeding edge homeserver, sign-up doesn't work yet. So it's not quite in grandma/grandpa territory yet... but once you're in, it most certainly is.
In the past I left sign up and setup to family and the element app and it has never worked smoothly. They don't understand the security key purpose or verifying clients.
So to answer your first question, yeah you can invite grandma and grandpa but they probably will need some hand holding at the start. Once they are in, it just works™**. YMMV
Just install Element on iPhone or Android and make a matrix account from there, then add each other by the selected username.
You will only have to help your grandpa when you start to add secondary devices that need to be authenticated the first time with a QR code (for decryption purposes).
I have been using Element to text my wife for around 6 months (Android to iOS). I took the opportunity when she was getting notification issues with a proprietary app to convince her to use it. She loves it now. Clean interface, no ads, and voice call works just fine for us.
but apart from that I of course love the idea of an „SMTP approach“ to instant messaging :)
Meanwhile, we really need to get https://github.com/matrix-org/rust-synapse-compress-state/pu... hooked up in Synapse so that it automatically compresses state; I don't know why this never landed when it was written in in 2021 :(
* You get best practice audited (in the near future) end-to-end encryption
* You get decentralisation if your users want to host the stack themselves
* You get bridges to other systems (although not many FOSS voip bridges for Matrix exist today, admittedly)
* Similarly, you get an open standard API on which you and others can write bots/apps and other extensions to add new functionality, and benefit from the wider ecosystem
* You get some preexisting clients like Element Call to build on.
This is great! Jitsi has never clicked for me. I tried it a few times, and read about it, and was never excited about it. I just tried Element Call, albeit an empty room, and the UI feels right.
Has there been any movement on migrating away from long polling (#475)? I didn't see anything about it in the announcements or comments despite such a big milestone
It might also get changed as part of Sliding Sync refinement.
It'd be nice to bring matrix to microcontrollers and "retro" computer platforms.
I had 68000, 8086/88 and 6502 in mind.
(Yes, there is a the Jitsi Meet Community Instances that are still with the original full functionality that people can access- If they search for Jitsi alternative community instances on a search engine, which isn't something most will be aware of for your average user...)
Having been a user of the Jitsi Meet Electron App even on Linux and Windows(along with the web accessible variant)- and having followed closely the efforts by Firefox on RTCRtpScriptTransform (formerly webrtc insertable streams) mainly to support Jitsi E2EE calls- has been interesting to watch.
Though now with the new Google/Meta/Microsoft account requirement/non anonymous aspect, all of that minus the community instances web interface sites are of use to me now.
If i follow fully, Element Call in a way replaces Jitsi Meet's original capability- though E2EE encryption doesn't have to be "Turned On " in a Chromium based browser - and Element Call isn't yet something that can be launched from a desktop app on Windows or Linux in a similar user-friendly way (I see the Vector IM client is a thing)
- The main difference being for using Element/Element X, one new from the Jitsi Meet ecosystem /usecases i've mentioned, will have to make an account- and for your average user who cannot easily learn about and deploy a server and how to host that on their own- but is looking for privacy- it is trickier to judge and will be more steps , to achieve anonymity from a privacy perspective.
I see the P2P effort is another step towards this as well
I'll likely try to use Element Call and point those I know who use tools like this towards it due to Jitsi's recent changes - as Element Call's web interface is user-friendly enough, and that is the main challenge. What Element is outside of that- and how it integrates into everything and how to use it in the same way in it's other forms- will be interesting to introduce, as those are developed further.
- Stuck notifications [1]
- Being unable to decrypt messages (from certain users sending from certain devices; some people can read them, others can't)
- No proper notifications for old threads
- Various GUI instability bugs: chats that are suddenly empty, text suddenly randomly overlaid on other text, randomly missing scroll bars, etc.
It's been good to us. Note that our primary comparison point is IRC, not Slack, mainly because of our project roots of openness and widely distributed contributors. So it's a serious workplace, but an atypical one. Encryption has periodically caused problems. Leaving the company and maintaining access to your communication seems to be a bit painful. Search is not great. Threads are tolerable. Inviting outside people in works well. Abuse is vastly more manageable than it was on IRC. Performance is mostly good. Both Element Web and Element on Android are very usable. Notifications are a little wonky, but I don't rely on them much so I can't really gauge whether they're good enough if you do.
You're comparing a Matrix server operated by someone else (Element) with an IRC network that Mozilla operated themselves. How does your Matrix setup compare to an IRC network operated by a dedicated team like Libera.Chat or OFTC?
It's really nice to see him answer in a calm and mostly objective tone while people are blaming him for shortcomings and problems of all the different Matrix-related software projects. Admitting to failures and problems that Matrix definitely has, while still defending the overall project and giving insight into the status of MSCs/projects is admirable.
Thank you Matthew. I wish you an endless amount of patience for the future.
Don’t think there’s a live stream. But there might be a recording afterwards.
Count yourself lucky for not having an undeterminable, unfightable legal threat over your head from a $800bn mega corp.
> Gatekeeper platforms will have to:
> - allow third parties to inter-operate with the gatekeeper’s own services in certain specific situations
> - allow their business users to access the data that they generate in their use of the gatekeeper’s platform
[1] https://edri.org/our-work/why-chat-control-is-so-dangerous/
Lack of interop has been a huge problem that’s also gotten worse over time. To address such a broad and complex issue with general legislation is not easy, we still have to wait and see if it works as intended.
open-wa, Baileys, invidious devs are all in the dark as to whether or not they're Scott free from their respective C&Ds due to this legislation.
I wonder if they were serious about it, or if it was just vaporware (a la Musk) to get more funding. It has been announced years ago, they raised millions and they have big governments contracts. WTF is going on?
a) p2p matrix works pretty well as a proof of concept - you can try it from https://arewep2pyet.com; it’s not remotely vapourware.
b) funding has been spent on core Matrix dev and making Element kick ass (cf Element X)
c) an awful lot of very large Matrix + Element deployments (eg LuxChat in Luxembourg, Merkury 2.0 in the Polish MOD, many German local authorities) decide to run Matrix deployments without routing a single $ to support development at either the Matrix Foundation or Element, such are the joys of the Apache license.
As a result, we’ve had to pause some of the longer-term R&D. I very much hope that P2P will resume (especially if someone explicitly funds it) and the team is still at Element (but working on sliding sync and account portability). But right now we’re focusing exclusively on fixing the core problems in both Matrix and Element which others in this thread are complaining about. Hence Matrix 2.0, and Element X.
I also do appreciate the Apache license.
Sorry for the vapourware comment. It's hard to tell these days. Was the Google self-driving car thing vapourware? It never went anywhere.
I know that pinecone worked as a PoC. I ran a node for years, for fun. It's just that it always felt like the progress was crazy slow. The yggdrasil demo was what, 4 years ago? How fucked up will the world be in 4 years? Not that it's your fault.
And I care about matrix. I was the one who suggested arewep2pyet.com, I made 1-2 friends use it (matrix), I made 1-2 coworkers use it.
Also, zulip threads now or I riot.
Well, it goes somewhere: people are actually being driven around Pheonix, Arizona [0] using those cars. It's just that it is not named "Google self-driving cars" any more: it was spun into its own company, Waymo [1].
[0]: https://waymo.com/waymo-one-phoenix/
[1]: https://waymo.com/
Shame you can't categorise Zulip streams like you can with channels on Discord though (something that's also related to the IRC/Matrix vs. Discord conversation/content discoverability thing mentioned elsewhere here).
(I mean by society not new vector)
Really hoping Full Matrix p2p will happen. Its would be such a cool capability.
Governments wasting so much time and money on so much nonsense software but we can't support and open communication tool.
I'd suggest you make sure there's a link to that page 1) somewhere in the menus 2) somewhere in the footer. I'd also make a link to the membership page from the "About" page, and a link to the "Support Matrix" page from the "Membership" page, so that people who manage to find the Membership page but aren't ready to become members can support via patreon / whatever.
Thanks for all your work on Matrix! Looking forward to seeing the improvements coming down the line!
This is extremely sad. They're large tax-funded organizations and should definitely contribute to priorize the needs they definitely have.
Ironically, there might be voices pushing for a move to a proprietary solution they would actually have to pay to use, and have much less control over.
Sorry for the bashing, but I got bitten once, and I see that nothing has changed in substance, so take this as a friendly word of advice: study the history of Matrix, its protocol, its implementations before committing to it seriously or for your org.
You and I exchanged 1:1 around 8 years ago on MatrixHQ, when this whole thing was still somewhat new but the cracks were already apparent. Then, the room was crumbling, federation wasn't reliable at all, the overall performance for basic use cases was terrible and self hosting on affordable hardware would mean not federating with matrix.org. When I brought those observations up, you (kindly and approachably) promised that a new design was in the works and weeks away (something about fanning out, or whatever). Fast forward to today, all the points above still hold true in 2023. From a distance, Matrix has had synapse, dendrite, better synapse with workers, conduit, new components added left and right (sliding sync being the latest), and I'm not even talking about the community at large, taken hostage of the uninterrupted pace of breakage. And I couldn't resist laughing out loud at fosdem when hearing this year once again that we are weeks away from what we were promised 10 years ago.
I am sorry to be blunt, but I've never seen a project of this magnitude lacking so much in structure and focus. I admit that I regressed to being a complete outsider, and from my perspective on the sidelines it looks like this has no end in sight: Matrix is getting more and more complex and hard for hobbyists to deploy at a small scale/local level, New Vector's hosted instances have a de facto quasi monopoly on the user base and on the protocol implementation. I'd be curious to know how this dissonance fits in your mental model, because this insane technical complexity was sold as the price to pay for not letting servers hold too much network power.
Anyways, I don't doubt that this message will be met with a well written and kind reply, listing a bunch of fringe projects and ongoing initiatives to tackle this or that aspect of the same old problem. See you again in 10 years, maybe :)
I'd be first to agree that Matrix hasn't evolved as rapidly as it should, and we (specifically me) could and should have been more focused. Building out future-looking projects like P2P Matrix or Low Bandwidth or Third Room has helped inspire folks to build on Matrix, but it's come at the expense of focus on the core spec & implementations. And that's one of the main reasons (besides funding) that we've shelved them for now. It's also true that 8 years ago, everything was flakey as hell.
However, whether you like it or not, we fixed it. The federation problems were resolved before the time of Matrix 1.0 back in 2019, and since then we've focused on making everything go fast too - e.g. Faster Room Joins in Matrix 2.0. We experimented with fan-out routing back in 2015 or so (as yet another future-looking experiment), saw it had potential, but at the same time saw that empirically full-mesh federation was good enough for the vast majority of cases. So we focused on fixing things like consensus bugs and lazy-loading joins rather than premature optimisation on full-mesh routing. Meanwhile P2P Matrix took over better-than-full-mesh routing, but practically speaking it simply doesn't buy enough to prioritise it relative to other stuff, and it certainly makes Matrix (even) more complicated.
> Fast forward to today, all the points above still hold true in 2023
This is demonstrably false, and it's particularly obtuse to be whining about it in the comments on a post on Matrix 2.0 which is literally dedicated to spelling out how Matrix has improved. Your complaint seems to be simultaneously that we've improved both the implementations (e.g. scalable Synapse) and the spec (e.g. sliding sync, removing Matrix authentication in favour of OIDC)... while complaining that the project hasn't improved since 2015. Personally, I think it's an extremely good thing that the new APIs start off as separate projects, which can evolve and release as rapidly as possible and benefit all homeservers, before being embedded into Synapse or whatever once they're stable. But it's also easy to spin this as "new components added left and right" as if it were a bad thing.
> New Vector's hosted instances have a de facto quasi monopoly on the user base and on the protocol implementation. I'd be curious to know how this dissonance fits in your mental model, because this insane technical complexity was sold as the price to pay for not letting servers hold too much network power.
There are >65k servers on the public network, of which about 2000 are run by Element (New Vector). All the biggest deployments (100K+ users) aren't run by Element (other than the matrix.org server), given the most people who pick Matrix do so because they want to run it themselves. Community services like Mozilla/KDE/GNOME are tiny by comparison, and they only end up hosted by Element because those projects don't care about self-hosting and would rather someone else runs it for them. They could equally well host with providers like https://etke.cc/ or https://ungleich.ch/u/products/hosted-matrix-chat/ or https://federated.computer or whoever; nobody's forcing them to use Element.
> Anyways, I don't doubt that this message will be met with a well written and kind reply
I'm sure you're a lovely person, but you come across as a complete asshole here, imo. Sorry you feel let down by Matrix; meanwhile most other people on the thread seem to think it has some merit. shrug
First of all, thanks for acknowledging the lack of focus of the project. I am happy to read that this is something being addressed, less so that the prime motivator for this is a lack of funds.
To me (and, I guess, to most), something like Matrix, which aims to become the broker of all our messaging needs, has an implied mandate to put reliability and consistency before all else. Your users' relationships, and sometimes, life? can be put at risk because of unreliable message delivery. Chasing new features and niche use-cases before the basics are covered by a sound theory and a stable implementation has always struck me as short-sighted: you can be sure that users burnt by over-promise and marketing hyperboles will never return. Enthusiasts and early-adopters who attempt to embark their circles onto the non-proprietary protocols exactly have one shot at it and a very high bar to pass. Contributors like yourself should also be aware that this "overly-optimistic attitude" insulates yourselves from potential contributors, in particular among the "theory/hard-problems loving" crowds, who are generally more suspicious about the kind or projects they engage with. That same crowd that could come to help with the interminable reliability issues.
> However, whether you like it or not, we fixed it. The federation problems were resolved before the time of Matrix 1.0 back in 2019
> This is demonstrably false, and it's particularly obtuse to be whining about it in the comments on a post on Matrix 2.0
You see, this is the kind of attitude that gives me little hope for the future success and adoption of Matrix amongst the general public. No later than a few days ago, using Element X on the client side, from a New Vector-managed account, sending a message to a(n up to date) federated server took 9 hours. I could top that up with many other issues with threads, spaces, client glitches, … but that's not the point. This is simply unacceptable for something branded as a mature product/protocol on par with the commercial offerings. I am not the only one reporting such issues and their reoccurrence. Every week, people on self-hosting forums give-up on Matrix for this or other reasons. Things are not going nearly as well as you make them to be, and whether you are honest and in denial, or malicious for the sake of protecting your business/career doesn't matter to me. What matters is that there is an end in sight to this and Matrix 2.0 isn't it.
> There are >65k servers on the public network, of which about 2000 are run by Element (New Vector).
I'm curious about the fraction of active users whose account is hosted by a NV-operated server vs the rest (as defined by the active users of servers federating with those). A vast majority of the Matrix accounts I see in the wild are hosted by NV (and I admit that my perception is skewed due to my involvement revolving around the opensource communities).
> I'm sure you're a lovely person, but you come across as a complete asshole here
I am critical of a work of love of yours, to which you dedicated a decade+ of your life, I can relate to the feeling. But for the record, my admiration for your dedication and availability answering messages here is legitimate.
Wow. Not interested in winning the war this way.
There's Element's Chief Product Officer & Managing Director presenting at the European Police Congress[0]: https://mastodon.matrix.org/@element/110310853505977058
And a quick post about their excitement at participating in this conference: https://mastodon.matrix.org/@element/110304013472307767
I mean, whatever, it's just doing business, right? When someone criticized this, they gave a snarky, surprisingly unprofessional response: https://mastodon.matrix.org/@matrix/110334695988903649
Dunno about anyone else but my trust in them is just gone. I'm not saying you have to care (and I don't care if you do). I'm sharing this for the people who do care, and don't want to use "secure messaging" made by people who actively market to and collaborate closely with law enforcement and governments.
The problem is if they give those orgs a backdoor
What more do you want?
You described the goal, yeah, that would be pretty wonderful, like unicorns, flying pixie dust and other things that don't seem to exist as of yet.
And I don't use them. I grab the posted sources and use them on hardware I physically control. If (I'm not, but I do care about my privacy) I was someone that was being pursued by one or more governments/well-funded private actors, I wouldn't use any communication platforms hosted by others.
As the old saw goes: "Three can keep a secret. If two are dead."
What’s wrong with helping organizations including law enforcement encrypt their communications?
Because they are public servants and We the Public should have ways to be able to decrypt those communications.
Secret meetings and intelligence agencies are what leads to totally avoidable wars.
https://community.qbix.com/t/transparency-in-government/234/...
I would also be excited to find paying customers when trying to build free software.
In any case I get the feeling if I were to avoid technology built by companies with government contracts I would be left with my retro 8bit machines. Lets be pragmatic.
I your distrust of law-enforcement so strong that any company that they purchase from must be untrustworthy? Or is there another reason that this bothers you?
It's not that govt/LE uses Matrix. It's that Matrix/Element actively seeks out partnership with govt/LE (to the extent of presenting on stage at a police conference), suggesting they are particularly interested in a relationship with authorities and personally sympathetic to these authorities, at the management/director level.
This is not a position I expect from an organization that is determined to build secure communications potentially used by activists, people living in oppressive/authoritarian regimes, etc... If I was a member of such a high-risk group, I don't think I would be putting my faith in this org to produce something I can actually consider truly secure.
If I were in such a high-risk group, I wouldn't be putting my "faith" in anyone. Rather, I'd personally make damn sure my sensitive communications couldn't be spied upon.
I'd also point out that repressive/authoritarian regimes don't care about the rule of law and/or personal privacy, so they can just come to your house and take you away without any proof that you're working against the state.
What's more, they can also confiscate any systems/devices you (or others) may have in an attempt to discover the identities of your "co-conspirators." And they can torture you to give up those names, even if the names you give up aren't actually involved at all.
You seem to believe that there's some sort of magical software that can keep someone safe if a government or well-funded private actor wants to get you. News flash: there is no such animal.
If the problem is that we're not trusting $LE_GOV_AGENCY to use the technology because it may hide abuses of authority, the root problem is that we can't trust $LE_GOV_AGENCY for reasons external to their choice of communication tooling. And that's not a problem I think we can solve with technology alone.
Plus, 'We can't allow X/Y/Z to use encrypted chat because they could do awful things' is a bit of a double-edged sword I personally don't feel comfortable wielding lest it be wielded against my own 'high-risk group'.
Disclaimer: SRE at Element, but I've been running a Matrix homeserver of my own since before I joined the company
Hopefully, eventually, the rest of the world will wake up to the fact that sleepwalking into using Teams and Slack (or WhatsApp) is a catastrophe in the making. It's only a matter of time before someone publishes a torrent of every line of Teams scrollback or Slack scrollback for some high-profile organisation; the breach has probably already happened; the breacher is just waiting for the best strategic moment to drop the information bomb.
But until then, Element makes payroll by selling to folks like the French and German Governments, the United Nations, etc. They pick Element because it is high trust, and they can audit it and run it themselves. And there is categorically no way that I or other senior management at Element would destroy the company (let alone Matrix) by breaking that trust by letting someone undermine its privacy.
Which is why we're in the surreal situation of on one hand selling to the UK Government, while also frantically criticising them for the catastrophic idiocy of the Online Safety Bill: https://element.io/blog/the-online-safety-bill-an-attack-on-....
Separately: I'm literally years overdue in publishing Element's internal ethics guidelines publicly, which spells out precisely who we do and don't do business with. For what it's worth, the high level summary is:
* We don't sell to criminals (under UK/US/EU law)
* We don't sell to sanctioned (by UK/US/EU) or abusive govts or organisations
* We don't sell to orgs who explicitly encourage use which goes against our terms of service.
Most western governments (including their police forces) do not fall into this bracket.
Thank you for the reminder. Based on that, are you claiming that Matrix servers/clients are insecure or have government back doors?
If so, what evidence do you have that supports such a claim?
If not, why should I care? I'm not a fan of many organizations/individuals. Should I survey them all to create a database of software those folks use and refuse to use any software they use? What value would I derive from such actions?
Unless you're claiming the former (evidence please), what difference does it make? I use Matrix servers/clients only on hardware that's under my physical control. And I use free (libre and beer) versions of Matrix implementations, so I'm not even giving the developers any money.
Or are you making the argument that LEOs shouldn't have access to secure communications protocols?
I guess I'm confused because I don't understand what the issue might be. If you'd elucidate, I'd much appreciate it!
You don't have to care, and like I said, I don't care if you care or not. I'm just making people aware, because some people _do_ care.
It's nice that you personally use matrix on hardware under your physical control. That's rare. The vast vast majority of matrix users have their account on matrix.org and are using Element. I'm glad to hear you are using alternative server and client software.
I addressed the other questions at https://news.ycombinator.com/item?id=37602089
>>I'm not a fan of many organizations/individuals. Should I survey them all to create a database of software those folks use and refuse to use any software they use? What value would I derive from such actions?
Why are those useful questions? Pick some software -- any software in wide use -- and you'll find that someone you find objectionable is using such software.
That includes OS's, browsers, databases, web/application servers, BIOS firmware, chat clients/servers, etc., etc., etc.
If using software that's also used by someone whose actions and/or rhetoric are objectionable to you is "capitulation" to those with whom you oppose/disagree, then you should power off all your electronic devices, smash them into little pieces and never look back.
But (IMHO, at least) taking such a position takes "guilt by association" to ridiculous extremes, especially when talking about open source software that can be audited and modified to suit your purposes.
Don't want your sensitive information shared? Don't share it on platforms not under your control. Full stop.
I'll say it again[0], "three can keep a secret. If two are dead." Food for thought, no?
And moreover that they used this money for something that you do like?