Hacker News Reader

> The fact that we operate a strictly optional logically centralised directory service to look up matrix IDs by email address or phone number. I really wish we'd never added this to Matrix; it causes way more grief than it solves

(Not a Matrix user so maybe help me understand here.) I think the reason why I have about 30 contacts (instead of like 3) on Signal is due to phone number discovery. It's not decentralized, which is of course subject to criticism, but exchanging & verifying public keys between O(N^2) people hasn't "taken off" in the X decades since it was invented.

> we'd have been better off waiting off for someone to build a decentralised keybase service and use that instead.

Do you have an example of any mainstream product that has ever achieved that, without either being partly centralized? Decentralized identity is really, really, really hard, and mostly the layer-8 implementation (discovery, name-squatting, impersonation, abuse, spam, key-management and exchange, etc).

And it is also the reason why I'd never considered signal for anything.

because it makes it easy for you to find contacts?

I do not want people to know I use X when they have me as a contact.

I also do not want my account to be tied to my phone number at all, because my phone number is tied to a contract.

well, keybase got quite far - it could have been decentralised but they chose to centralise it for commercial reasons. but yes, decentralised identity is hard, hence us cutting that corner (and then getting abuse about being too centralised…)

Every Matrix protocol client I've ever tried, including yours, seems to connect to your "strictly optional" centralized directory service. I've never once wanted to use it, and, despite your claims of it being optional, 100% of the times I have logged in to a Matrix instance, via the web or via an app, my computer has talked to your computer.

Something's broken here.

See https://news.ycombinator.com/item?id=29263797.

Amazing work. Bravo. I'm very much hoping to see P2P Matrix arise. That is no small task, I realize (and I'm grateful to those who have contributed to this). What are you thoughts on that aspect of the ecosystem?

P2P Matrix is a moonshot, but it's coming together really well. The only bits left at this point are:

* Making Pinecone resilient to attacks. It's working well for clear weather scenarios, but we need to properly try to break it and get it audited. Need to test it at scale too.

* Improving Dendrite so that it's more stable and efficient clientside

* Metadata-protecting store-and-forward, so that clients can pass messages to each other without needing to be online at the same time. You can cheat and run a serverside P2P node to relay, but then it stacks up metadata - whereas half the point of P2P is reduce Matrix's metadata footprint.

* Account portability. Currently each node is a separate account; need to support the same logical account being replicated over multiple nodes.

* Better than full-mesh routing. This should be pretty easy as we can just follow Pinecone's topology to route traffic between nodes rather than having each node try to simultaneously post to every other node in a given room.

You can get testflight & APK builds from #p2p:matrix.org if you want to experiment.

Arathorn, thanks for responding to comments here. Public discourse comes in all flavors and I appreciate you taking time to respond to the queries and comments. :)

AFAIK, there are 6 server implementations of Matrix protocol: Synapse (Python), Dendrite (Go), Conduit (Rust), Construct/Charybdis (C++), Plasma (Elixir), Jeon (Java). Are there any more that I have missed? Which ones other than Synapse are most mature? An unbiased comparison post (that remains updated) would be really useful.

A second question is: are the APIs between a homeserver and the frontend defined and standardized? I've noticed this in the ActivityPub ecosystem where it is usually very easy to use one frontend with another backend. Are matrix servers also written in the same way?

Not OP,I'm just a happy matrix user.

Yes the matrix spec is all standardized. You can use any client with any homeserver.

There are a few other server implementations (polyjuice in Elixir https://gitlab.com/polyjuice/polyjuice_server; https://github.com/clecat/ocaml-matrix in OCaml; the abandoned mxhsd in Java) - but the most stable ones alternatives to Synapse right now are Dendrite (Go) and Conduit (Rust).

Matrix's API surface is big, and servers deliberately handle all the heavy lifting in order to make clients trivial to write, so writing servers is not trivial. But Dendrite & Conduit are both very usable and making good progress right now.

And yes, the APIs are all standardised (while also constantly evolving) as per https://spec.matrix.org :)

Thank you! Those are extremely important points, and I'm glad to hear it. Rambling from a madman in the desert, so please feel free to ignore me or tell me buzz off (and, if so, it was an honor speaking with you!):

* This might be a worthy tool for simulating aspects of scale: http://imunes.net/

* I assume power usage is a non-trivial concern (looks like yall have been worried about this too). I've tried my fairshare of P2P apps on mobile, and this can be a dealbreaker for many folks. I assume this will be something that has to be dialed in on mobile clients (and probably provides options to the user). I'm sure there are many trade-offs to consider, and I'm hoping logical accounts will provide significant advantages here. Are you feeling confident this will succeed?

* The Store-and-Forward problem looks like a really big deal. Defaulting to friends (for those folks who are lucky to have them) or a chosen subset to proxy seems reasonable. Having the clear ability to pick out who we trust here (including volunteer networks) seems solid (I'm a huge fan of Retroshare's F2F concept). I take it that, ideally, with logical accounts and excellent NAT traversal (also a sore spot usually), one's own cloud of machines will obviate some of the desire to run anything in a datacenter (and even perhaps make friends a fallback). I'm excited to see Dendrite and Conduit grow!

* As part of that portability, at least to my ideal, I'd be able to join my network from scratch using a password hashing algorithm for the private key, as otherwise I may be reliant upon secondary networks to move keys. This is often one of the first applications I want to install on a system to bootstrap myself in.

* Pinecone's Topology looks like a good idea (Yggdrasil is dope too). That metadata problem may still raise its ugly head here. Do you predict the use of onion routing, at least with Tor and i2p, will be relatively easy to integrate in addition? Do you foresee Matrix enabling its own onion routing some day?

* If you had to take a wild guess at the first stable moon landing, when do you think that might occur? Do you think 5-years may be acceptable?

Lots of interesting stuff there - thanks :) We're using https://github.com/mwarning/meshnet-lab rather than imunes.net for network simulation currently, but will take a look.

Power usage is looking pretty positive so far; as long as we route the Matrix traffic over the routing topology rather than going full-mesh it should minimise radio usage (the main battery suck, other than screen).

For store-and-forward, honestly using P2P Nodes as intermediaries is an okay approach other than exposing metadata to them. Our plan in the longer term is to switch to loopix-style mixnets to obfuscate the store and forwarding, a la nym.

In terms of joining the network by deriving a private key from a passphrase... yup, that could be cute, although slightly terrifying in terms of the risk of weak passphrases :)

We're hoping to get the P2P network stable in the coming year (although we were also aiming for this year originally :P)

You are exceptionally polite speaking with lunatics like me (I've seen that multiple times from you). I'm [[grateful]] for your candid responses. In case it may be useful, I will continue thinking about it here: https://philosopher.life/#2021.11.17%20-%20HN%20Log%3A%20Ara.... You already know, but it is my duty to acknowledge that you work on behalf of Humanity and quite a bit may ride on your success. You have a difficult ship to steer without enough resources, and I wish you had military-level funding for what is ideally a military-grade project. If you ever directly need something from me, please let me know. Thank you again for your time and willingness to listen to me, sir.

I just tried logging in to Element.io with an account on my own homeserver and I saw several requests to vector.im before the privacy policy popup, including what seems to be some kind of token exchange.

I don't think the intent is malicious, but the system is definitely leaky. My homeserver config doesn't mention vector.im (perhaps it's a hidden default somewhere?), I checked.

From a protocol standpoint, no identity server is necessary. From a practical endpoint, all feature-rich clients (the ones I know support e2ee) seem to send requests to vector.im. Matrix is decentralized, but a lot of data still seems to get leaked to a centralized place by accident from my short experiments. It's not a great scenario, even if I'm fine with a few requests to the 3PID server every now and then.

Client issues are ecosystem issues if all usable clients suffer from them.

> I don't think the intent is malicious, but the system is definitely leaky. My homeserver config doesn't mention vector.im (perhaps it's a hidden default somewhere?), I checked.

IIRC, identity server is something primarily set on the client (element.io here) rather than the homeserver.

It really needn’t and shouldn’t ping an identity server as part of the sign-in process (esp before/without privacy policy agreement)

FWIW it seems the issue is that there’s a fallback process for identity server; if neither your account nor your homeserver has one set, it defaults to the client default. if you host your own Element-web instance you can change this.

https://github.com/vector-im/element-web/blob/develop/docs/c...

Or check out ma1sd. You don’t need to enable all the bells and whistles if you don’t want to.

So I've just double-checked this by opening https://app.element.io in an incognito tab and looking at the network inspector. What I see is what I was referring to in the grandparent as:

* Element has to query your server to find out what authentication mechanisms are available before you log in. Therefore if its config points to the default matrix.org homeserver, you end up pinging that every time you log in, even if your homeserver is elsewhere. The solution for now is to change your default homeserver in the config to avoid this.

The same applies to the identity lookup server as well as the homeserver: the app checks the one in your config to ensure that it exists before the user tries to log in on it. It does this with a single request (a GET to https://vector.im/_matrix/identity/api/v1). There is no token exchange there.

The workaround is to configure the app with the correct default homeserver & identity server (if any) for your deployment. The underlying bug (i.e. don't validate the config until the user tries to log in) is https://github.com/vector-im/element-web/issues/11655, and it simply hasn't got to the top of the list yet; we've been focusing on narrowing the UX gap between Element & Discord/Slack/Teams rather than reimplementing the login flows. As you can see on the bug, it's been picked up off the backlog 9 days ago and is currently being worked on.

Separately, I'm seeing a related bug that Element Web's guest mode is incorrectly kicking in on the login page. Guest mode exists so you can deeplink straight to something like https://app.element.io/#/room/#matrix-dev:matrix.org and read scrollback without needing to register an account - but it looks like we're spinning up guests on the default homeserver needlessly on the login page. This is https://github.com/vector-im/element-web/issues/11533, and we've put it on the radar too.

TL;DR: this is slightly leaky, but is it really any different to Chrome's default config loading up Google as a default homepage? Or Firefox loading up Mozilla.org? Or Thunderbird loading up Thunderbird.org? Eitherway, we'll fix the behaviour, but PLEASE understand that this is a million miles away from the bogus "central authority can have influence on another server" smear from the original post.

> TL;DR: this is slightly leaky, but is it really any different to Chrome's default config loading up Google as a default homepage? Or Firefox loading up Mozilla.org? Or Thunderbird loading up Thunderbird.org? Eitherway, we'll fix the behaviour, but PLEASE understand that this is a million miles away from the bogus "central authority can have influence on another server" smear from the original post.

I understand the whole "central server" stuff is bogus.

However, I consider the forced homepage opens every browser seems to do today an invasion of my privacy, especially with the way these pages often includes trackers. Mozilla is especially bad at this; I've had to disable checkboxes on all my devices to make sure I don't get their stupid VPN ad, which is tracked with Google analytics and which addons can't properly touch because certain Mozilla domains are exempt from addons (rightfully so!).

Google knows when I open Thunderbird. Google knows when I update Firefox. Google knows when I do anything in Chrome. Microsoft tracks even more than Facebook when I accidentally open Edge because I was foolish enough to click on a help icon in Windows 11. Element knows when I log in to Matrix. All of this data collection is completely unnecessary, and while this comes from a sick hunger for data in Mozilla's/Google's/Microsoft's cases and a configuration quirk in Element's case, it's still a worrying development in my opinion.

I know you folks are working hard to fix things like these and I originally wanted to call balogne on the claims several people made in these threads, but I couldn't because their observations about the identity server were right (though their conclusions were strongly overstated).

I'm glad that this is on your radar and I'll happily keep using Matrix while this is being fixed in the background, so don't get me wrong here. I love Matrix and the possibilities it represents, and I hope the clients and features will continue maturing so that I can recommend my friends and family to get it.

Matrix is a lot better than every proprietary service, but it's poor XMPP compliance currently makes it impossible to have essential features like encrypted communications or adding Matrix users to multi-user chats.

Unfortunately XMPP<->Matrix end-to-end encryption is a contradiction in terms. You have to speak the same protocol from end-to-end, otherwise you'd have to break the end-to-end encryption to translate from one protocol to the other. The closest you could get would be to run the Matrix<->XMPP bridge clientside alongside your Matrix client... but at that point you might almost as well be running a multihead messenger that speaks both Matrix & XMPP.

In terms of adding Matrix to MUCs; i believe we're working on it in Bifrost, which is in active albeit slightly sporadic dev: https://github.com/matrix-org/matrix-bifrost.

> Unfortunately XMPP<->Matrix end-to-end encryption is a contradiction in terms. You have to speak the same protocol from end-to-end, otherwise you'd have to break the end-to-end encryption to translate from one protocol to the other.

You'd only have to speak the same encryption protocol. There already exists standards like the Signal protocol (on which OMEMO builds) or OpenPGP (which many XMPP clients support besides OMEMO). If messages can cross the border, so can public keys and encrypted messages.

Unfortunately this isn't true. Olm (Matrix's encryption protocol) is the same as OMEMO, which in turn is the same as libaxolotl. In fact, the OMEMO XEP even recommended Olm as an implementation for OMEMO at one point due to licensing concerns with libaxolotl/libsignalprotocol!

But the fact that both protocols use the same E2EE encryption protocol doesn't help you with interoperability - given that you have to break that E2EE in order to convert between Matrix & XMPP; either on the server (ugh, defeats the whole point of E2EE) or on the client (ugh; you might as well just have a multiheaded messenger).

I'm not sure I understand. If in fact the same underlying encryption protocol is used, why can I not retrieve the public key of a Matrix user, encrypt the message and send it? Nothing here requires the server to break E2EE.

Because the message format is different. In XMPP, the message is an XML document according to the message schema. In Matrix, it's JSON.

You can exchange the private keys, but using what format? Once that's done, you can encrypt XML on one side, but Matrix has to convert it into JSON on the other (in its own, separate format with its own fields). This can be done on the server side after decryption (no more E2EE), or on the client side (after decryption). In the latter case, you're not running a bridge anymore, you're just running an ad-hoc client for XMPP inside your matrix client.

In XMPP, encrypted messages can either contain encrypted XML

> <message type="chat"><body>Hi there!</body></message>

or raw text

> Hi there!

In Matrix, it's always encrypted JSON

> {"type": "m.room.message", "content": {"msgtype": "m.text", "body": "Hi there!"}}

It would be possible for Matrix to indicate that a room is a "raw room" and that clients should send and expect raw messages in that room - losing a few features on the go. Such raw room could be bridged over to XMPP fully end-to-end-encrypted (with small changes on either side to adjust for the differences between OMEMO and OLM).

But I doubt anyone at Element will invest time into doing it and for the XMPP side it's much more work to do it "on their own" (by implementing parsing support for the encrypted JSON of Matrix as shown above).

Having raw text in the encrypted content is by the way still the normal way how things work in XMPP world and is supported by every client doing E2EE - because it is basically free to implement (barely any additional code needed).

Even if you could make Olm/OMEMO interoperate otherwise, the Signal protocol supporting client requires access to a server to pick up pre-keys. So you need to have access to the pre-key server of your correspondent or your correspondent needs access to the pre-key server you are on. The Signal Protocol is not generally suitable for this sort of thing mostly due to its complexity.

If you did something stateless like OpenPGP then you could do E2EE over bridges with no problem. The cost would be that each and every message would be complete in itself. So sending a single emoji would take several hundred bytes. That might be less important these days...

> but at that point you might almost as well be running a multihead messenger that speaks both Matrix & XMPP.

Yes please! There's sooo much work people put in the clients, and there are multitude of them for each network. I understand they are different and a specialized client may be better suited for unique protocol features, but I also think there should be more general clients with protocol plugins, as it was with Pidgin, Miranda and such.

> You have to speak the same protocol from end-to-end, otherwise you'd have to break the end-to-end encryption to translate from one protocol to the other.

Why is this any different from WhatsApp, which has a Matrix bridge? Or does that break encryption too?

I have no idea how the Whatsapp bridge works but it cant be end to end encrypted.

Matrix to whatsapp bridge can be encrypted, whatsapp bridge to whatsapp can be encrypted, whatsapp to whatsapp is encrypted. But you have to break the chain at some point between matrix and whatsapp.

Encrypted != End to End Encrypted.

Seems to be my understanding as well.