Hacker News Reader

>trading off for speed

If speed is a concern, why did you all stick with Synapse (essentially single-threaded due to the GIL) over moving to Dendrite? As far as I can tell, Dendrite is, for all intents and purposes, abandoned.

Because we didn't have enough people or cash to do a good job of simultaneously writing two servers, and as Synapse had gone into production across *.gouv.fr and other critical deployments, we instead frantically backported Dendrite's main novelties to Synapse - adding instead worker processes to Synapse so it could easily scale beyond the GIL: https://matrix.org/blog/2020/11/03/how-we-fixed-synapse-s-sc...

The hope was always that we would then get back to Dendrite and be able to invest in it and migrate over, but the cash situation got worse in 2022 due to Matrix being more and more successful (causing the available $ in the industry to be flow to integrators rather than the upstream project), and instead we had to essentially park Dendrite dev in 2023 other than for critical fixes.

Meanwhile, to try to fix the $ situation, we added Rust workers to Synapse as "Synapse Pro" to give customers a reason to actually route money to us as the upstream project, and nowadays Element is actually on a more economically sustainable path. However, at this point the likelihood is that rather than progressing Dendrite we'll instead look to add more Rust to Synapse and fix its resource usage. That said, others are of course very welcome to continue progressing Dendrite forwards, and I personally find it super depressing that we failed to progress both servers at the same time.

Matrix team is doing a solid job of running - Keep it up and keep eating the Slack/Teams marketshare up with competitive features and pricing. Additional business considerations like HQ location costs, tax liabilities, and talent pool availability on paper also affect what you have to work with. London tax, talent, and labor pay versus Austin for example.

Also I got your name wrong last time - I apologize for that.

> nowadays Element is actually on a more economically sustainable path

Good to hear. Keep up the good work.

It sounds like you were stuck between a rock and a hard place there. Hope the Rust integration goes well.

> the likelihood is that rather than progressing Dendrite we'll instead look to add more Rust to Synapse

I thought the goal of Dendrite was decentralization done right? Namely, the ability to run a homeserver from the very phone one is using the client on?

Dendrite did subsequently switch to powering the P2P Matrix work… which also got paused in 2023. We’re currently resurrecting it, but it’s not clear whether Dendrite will be the clientside server impl.

Been loving your well-placed comments for 5 years.

Keep up the great work - your team should be giving you a raise; you’re a great reflection on them.

What’s the best way to fund development of this stuff? I’m aware of donating to the matrix.org foundation, but as far as I can tell none of that goes towards funding server and client implementations since those are Element instead of the foundation.

https://kashmirlife.net/14-messaging-apps-blocked-in-jk-3163...

You guys gave up on the national security threat and caved.

Dont want authorities knocking my door down for using an app

Custom emoji also stood out to me as “who cares”,

when compared with the other concerns throughout the thread.

> This is nothing to do with state resolution; it's due to Synapse's implementation deliberately cutting corners on storage efficiency while trading off for speed.

It's specifically to increase the speed if *state resolution*. If it weren't for the convoluted state resolution system, there wouldn't be a need to store gigabytes worth of state groups in the database.

* https://element-hq.github.io/synapse/latest/usage/administra...

* https://github.com/matrix-org/rust-synapse-compress-state

Maybe there's a way to calculate state without state groups, but I sure don't see one that I can use if I were to run a matrix server.

> Not since https://matrix.org/blog/2025/08/project-hydra-improving-stat...

Simply fixes some of the many ways that rooms can explode or be bricked. Zero confidence that room brickings are totally fixed once and for all.

> There are MSCs for all of these now, and implementations are starting to filter through. The reality is that the project has been in a funding crunch since 2023 and we've had to focus on survival by prioritising stuff people pay for (i.e. big servers for govtech deployments) rather than custom emoji.

A funding crunch since 2023 yet those features have been necessary for many years before 2023.

The room state is cached to not need to recompute the current room state from the beginning of time.

You probably would do that even if there was no state resolution at all

> Simply fixes some of the many ways that rooms can explode or be bricked.

How many other ways are there? Afaik none is known

> Afaik none is known

Before project hydra people didn't know about the room exploit either. They just knew that rooms exploded somehow every once in a while.

Most people actually working on Matrix have been aware of state resets for quite a while. Hydra is just the name of the project which addresses them. There are 3 phases, of which the 1st covers the most serious ones; the 2nd and 3rd phases should drop next year.

As an analogy, it's not dissimilar to how Git has added various different merge resolution approaches over the years in order to come up with more predictable and more "do what i mean" algorithms (resolve, recursive, ORT, octopus, etc). It's slightly different in that a bad merge in Matrix feels very unexpected and problematic, whereas manually unpicking collisions in a VCS is just part of the territory.

> It's specifically to increase the speed if state resolution. If it weren't for the convoluted state resolution system, there wouldn't be a need to store gigabytes worth of state groups in the database.

No, it's specifically to increase the speed of state retrieval. One of the uses for that is state resolution, but it could equally well just be calling the /messages API or any other point you need to know historical state. But what do I know :)

> it could equally well just be calling the /messages API or any other point you need to know historical state.

State groups aren't really a thing for messages in a timeline, there are many easier ways of doing it, for example, simply storing the message sequentially (impossible in matrix though, due to the convoluted tree structure it uses)

But when it comes to state (where state groups are actually needed) who actually needs a snapshot of the state at literally every point in history? Any other messaging app just needs to know the current state and maybe also an audit log of the change history for audit log purposes.

In any sane messaging app (e.g. discord, slack, telegram etc.) there is exactly zero relevance in knowing the member list, room configuration, permissions and room title at exactly 2024-06-19T15:23:45Z. Who the heck cares??? Yet the design of matrix somehow makes this an integral part of every single operation.

Perhaps just watch the video and see that the proposed solution is just doing a temporal state table - just as Slack and Discord etc must have in order to know what the state of a room was at some point in time.

Slack and discord don't know the state of the room at any given time in the past. Show me anywhere in slack or discord where you can see the membership of a room at 2024-06-19T15:23:45Z. Or anywhere where you can see the historical profile picture and nickname of everyone in the room at 2024-06-19T15:23:45Z. You can't, because they don't know.

I am pretty confident that slack and discord know who has permission to read a message at a given point in time, which is all that state groups are achieving here.

I am pretty confident they only know who has permission to read it right now.

No, they all need to know who had permission to read a message which was sent at a given point in the timeline. This is all state groups is: a track of the room state at the point a message was sent: e.g. who has permission to read it.

I repeat: I am pretty confident they only know who has permission to read it right now.

It works like this: their server gets a request to read messages, and checks if the client currently has permission to read messages. If so, it reads the messages. If not, it returns a permission error. If you don't have permission, it doesn't attempt to analyze history to figure out whether you had permission in the past.

In fact, Discord is pretty explicit about not letting you have any interaction with things you don't have permission to view. It doesn't even tell you you don't have permission. It just silently removes them from your menu, and if you try to access them with a link, it tells you "unknown error".

> A funding crunch since 2023 yet those features have been necessary for many years before 2023.

But before 2023, the funding was going to things like solving state resolution, a VoIP system that was not dependent on Jitsi, getting rid of "could not decrypt message" errors, and so on.