Nation-scale Matrix deployments will fail using the community version of Synapse
mastodon.matrix.org
mastodon.matrix.org
To me it feels like they've ended up in a position similar to Docker Inc. where they've spent years of work and tons of resources building the standards and reference implementations, but missed out on the juicy integration and support contracts.
Docker has turned into the OCI standard and most Kubernetes deployment don't even run containerd anymore. The company has massively rate limited Docker hub and recently increased the pricing again, which has had much bigger impact for end users than this pivot by Element.
I really hope this works out as a funding source for future Matrix development, I don't see how its ecosystem can continue to be developed and healthy without Element.
I assume you mean moby, which was used via dockershim in Kubernetes until support was removed in v1.24.
I can't find a reliable source, but I believe containerd, followed by CRI-O and alternatives, are the common choices of container runtime for Kubernetes.
https://kubernetes.io/docs/setup/production-environment/cont...
I know from my own experience that making money based on an open source project is extremely challenging.
However, I don't understand why they frame their Pro version in this way and point to failure instead of success. I can't think of a better way (they will have to do the work :)). But it's already a good thing if they've won them as "Matrix believers", and they shouldn't scare them to get there.
This.
> However, I don't understand why they frame their Pro version in this way and point to failure instead of success.
It’s really trying to spell out the risks of freeriding, given the severity of the situation. But you’re right that it may well be too negative.
> I can't think of a better way (they will have to do the work :)).
Someone proposed a better positioning in the OP thread at: https://ioc.exchange/@troed/113843356547707182
Is anybody anywhere - Deutsche Telekom included - actually making money here?
Some massive organisation (government agency, healthcare provider, network of schools etc) says: "Wow, Matrix is great, we should run our own instance rather than be dependent on Teams/Slack/Workspace. I'll put out a public tender for suppliers to provide this for us!"
Then, all the system integrators who are geared up to sell big IT projects to large organisations jump on the tender and say "Sure, we can do this for you, and we're best qualified to do so because we're an enormous trusted system integrator with loads of people in the right geography, got all the right compliance checkboxes, and we do this sort of thing all the time. It'll cost you $xM a year."
The upstream developer (Element) then reaches out to the integrators and says: "Oh hi! Are you planning to bid on this contract? If you want this to be successful, you should use our commercial offering, as then we'll be able to use that $ to keep upstream development alive".
The integrator then looks bewildered, and says: "But... we've been using your opensource distro off github and it seems fine? Why would we pay you guys anything? The additional cost will just make our bid too expensive and we won't win, or it'll kill our profit margins. Sorry, we'll go ahead without you and maintain the software ourselves if needed."
Given the main paying customers for Matrix are big organisations wanting digital sovereignty, this means the available $ disappears into the integrators rather making is way to fund the actual upstream dev - which then ends up stagnating, stalled, and completely at risk - putting all of these projects at risk. facepalm.
A variation on the problem is where the massive org decides to do the project in-house rather than via an integrator... but then still chooses to YOLO and freeride and given it'll cost less than using the commercial offering from the upstream. I can list at least 4 national governments who have taken this route.
To solve it, options include:
* The buyer could mandate that suppliers for the tender have to use the commercial offering from the upstream product rather than freeride on opensource. This is probably the easiest solution, but seems staggeringly rare - the only instance I know of who have taken this principled stand are ZenDiS in Germany (https://zendis.de) and we are very thankful for it.
* The upstream could make their commercial offering more valuable than the FOSS offering - e.g. additional domain-specific functionality: what we've done with Synapse Pro.
* The upstream could make the FOSS licensing less permissive, to force integrators to contribute back. We already did this with AGPL Synapse in 2023: https://element.io/blog/element-to-adopt-agplv3, and it has helped a bit... but in practice, we just saw a fleet of major Matrix tenders go to system integrators who then apologetically cut Element out, despite AGPL - putting Element's funding for 2025 directly at risk.
Which is why we're putting out brutal blog posts saying "Look, big organisations: if you try to freeride on FOSS your projects will fail" - and providing a quantified explanation of the value that the commercial offering provides... so that we can then take the $ and hopefully keep the primary FOSS project on the road.
> but in practice, we just saw a fleet of major Matrix tenders go to system integrators who then apologetically cut Element out, despite AGPL - putting Element's funding for 2025 directly at risk.
I appreciate the transparency and you totally don't have to but not sure I fully understand this sentence, do you mind explaining that further?
If you look at companies like Red Hat, and Nextcloud, neither of these companies sold proprietary versions of their software. Red Hat sold licensed versions of their software, but they were always FLOSS.
Nextcloud's CEO talks about how selling a proprietary version of your code is bad for your customers and puts your company into a Catch 22 with itself. "Do we add this feature to the 'Community' or 'Enterprise' version?" or worse "What do we do when someone makes a FLOSS plugin that competes or is better than our internal version of the same functionality?"
There's a huge need for a good chat system to compete with the likes of Slack, but Matrix has bitten me several times and I don't trust it. In an alternate universe, they could have made Matrix work well for all levels of users and then sold customization and B2B deals with their customers leveraging the community, which has always wanted Matrix to succeed.
As for it being the best of both worlds, it depends what best means.
A company that wants to make a lot of money won't do well with this model. It's a slow growth model requiring a lot of hands on work that requires customers who are paying for features and custom installations.
It's labor intensive work that's quite the opposite of the proprietary software model, which has traditionally had a lot in common with the publishing or entertainment industry. Write a book, get paid for it for years.
More importantly, Red Hat was very profitable even when CentOS was a drop in replacement.
Matrix servers (and web servers and mail servers etc) are not as complex - and if anything you want incentives to make them less complex, not more. And the better job you do of making them easy to admin and support (ie the more scalable and stable they are), the less reason there is to pay the upstream for support but some system integrator instead.
Just my perspective but I think this was also perhaps a product of the time when Red Hat really started growing, back when linux was a little more rough around the edges. These days it seems linux has enough polish that some (many?) corporations are happy to just use off the shelf ubuntu and call it a day and hope nothing breaks too badly.
I think it has dawned on people that it is impossible to avoid using open source.
0: https://medium.com/@gordon.messmer/in-favor-of-centos-stream...
CentOS started as a fork, and new forks exist now.
I thought this was Zulip?
I'm about to fork Dendrite at its last ASL 2.0 version, because to hell with signing a CLA for Element.
I receive messages < 10 seconds. What's your setup?
Customers won't pay for development unless forced to and it creates a tragedy of the commons. No large commercial deployment will accept 5 year out of date broken insecure software.
Non paying users won't care that they can't use it commercially
https://stgraber.org/2023/12/12/lxd-now-re-licensed-and-unde...
They forced me off of it due to offerings they were no longer servicing. Told me I had to self host and export all my data. I attempted this and it never worked. I abandoned that server and my profile I used across many matrix instances (and somehow my matrix room continued to run without me hosting it, and without an admin running it).
I will never use nor recommend them ever again. They clearly do not know how to operate a business nor an open source project.
Surprise, this is probably the primary architectural feature of Matrix. This is precisely what it's designed to do.
Notwithstanding the naysayers that surface with peculiar predictability any time Matrix is discussed here, know there are people like me that are happy Synapse users (and operators, nearly 5 years now!) that are in your corner, rooting for you [and Matrix]. Keep fighting the good fight.
Synapse Pro is a compromise to keep the ship afloat. This is not a matter of greed, but a matter of keeping the lights on.
For what it’s worth, we have been frantically trying to keep all Synapse dev FOSS for the last 10 years - and the FOSS version will always be the primary project, where everything new happens (other than mega-scalability work). At least for as long as we’re able to work on it.
My hope is that Synapse Pro will be net positive for the ecosystem, as it will hopefully fund us to be able to work more on FOSS Synapse. It could also inspire more alternative homeserver work, which is no bad thing for Matrix as a whole.
Matrix is a cool idea but we now have like 1 server implementation that isn't somewhat gimpy, and its closed source... XMPP is much more compelling.
Exactly who should "require" these companies to implement theses standards? Some daddy government? Who would be forced to implement it? Just the big sites? What makes a site big enough? If it were required, it would be used to lock out smaller players.
Consumers could demand it, but we know the average consumer doesn't even know what a protocol is, or understand the different between a web browser and e-mail. Most don't even know how their phones work.
I don’t get why people want to me part of a community where the steward treats them in such a shitty way.
1. its preferable to a proprietary alternative. 2. its an open standard so there are alternative implementations 3. Very few people need the high performance alternative.
It might not be ideal, but what is the better alternative.
GrapheneOS dev account seems to disagree about the scalability issue being limited to very large instances. Due to the way how Matrix works, even activity on other servers can generate events at your server, particularly when you have joined busy channels.
Not a software engineer, but a server software that collapses at only 100 events per second on modern hardware sounds pretty bad to me.
Entirely separately: there are bottlenecks you hit if you try to run a 100K+ user server. Synapse Pro fixes those bottlenecks, and the idea is the $ derived from that can then fund the broader FOSS improvements, which otherwise are starved of funding.
Not only they are admitting that it is an exercise in futility to set up a medium-to-large Matrix deployment using an open source software stack.
100 000 people is not "nation-scale". It is small town scale. I am wondering if they intend Matrix ever to become truly mainstream instead of being confined to internal use of government organizations.
And it only gets worse from there: https://element.io/blog/synapse-pro-slashes-costs-for-runnin...
Synapse Pro was released only about a month ago. All these years they have been selling their solution to large organizations, despite it being hopelessly inadequate what it comes to scalability.
This really sounds like "haha you community members are stuck with a sucky version of our software and you're never getting the good one!". It's not exactly making me feel valued as a community member. And we shouldn't forget this is what put matrix on the map. This feels very bait-and-switch'ey.
Also pointing out the flaws in one's own software doesn't inspire a lot of confidence. I think it would have been a sounder approach to paywall real enterprise features rather than the core product. Such as SSO, auditing, legal holds, that stuff. Basically all the stuff that Microsoft Teams has in Purview. The community won't miss any of that stuff.
Ps I thought dendrite was supposed to be the next big thing? It feels like element is often jumping on new things like big rewrites such as element X instead of improving the existing products.
By making only optimisations that primarily support massive servers proprietary (for now), the $ from Synapse Pro can hopefully then directly fund the features and maintenance of core FOSS Synapse, including core optimisation work which benefits everyone. Concrete examples include:
* State Res v3, which hopefully should have algorithmic improvements to perf
* State storage, addressing the long standing diskspace consumption problem for state snapshots
* Finishing faster room joins so they are actually fast
* Shared retry hints on federation, to stop wasting time trying to talk to dead servers
* Better than full mesh federation, so small servers can participate in big rooms without melting.
EDIT: to spell it out, as it’s been misunderstood: Fixing these issues will likely happen directly in FOSS Synapse (assuming there’s any $ to work on them at all).
Meanwhile, FOSS Synapse already has Rust as Python codepaths and has done for years, and that will only increase over time. Only stuff for making workers scale to handle hundreds of thousand concurrent users will remain proprietary for now.
Dendrite was meant to be the next big thing, but we (obviously) didn’t have enough $ to work on both it and Synapse. So we focused exclusively on Synapse as of end of Dec 2023, and Dendrite is currently progressing on a best-effort (ie not funded work) basis.
> It feels like element is often jumping on new things like big rewrites such as element X instead of improving the existing products.
Well, Dendrite was a big rewrite like that. Which is why we’ve ended up improving Synapse instead (both FOSS and the new rust workers).
It doesn't inspire confidence that we have to wait for this to 'trickle down' to the free version.
And ok yeah I think it's a good idea to kill concurrent product development that does the same thing. As long as it makes the remaining product better.
So there's also a Rust version of FOSS Synapse? I wasn't aware of that. Even the official blog post speaks only of python for the community version.
> By making optimisations that primarily support massive servers proprietary (for now), the $ can hopefully then fund features and maintenance of core FOSS Synapse
Your own blog post about the pro version (and the one about forking synapse) basically said that you wouldn't be working on the community version at all anymore? Unless I misunderstood. But my takeaway from this is 'it's a dead duck, it's time to pay up'. Which is a bit worrying as this is the kind of price only viable for large corporations.
There is no trickle-down happening here: *these issues will be fixed directly in FOSS synapse, which is the core project*. They are nothing to do with rust worker implementations in Synapse Pro, and FOSS will get them first. The only reason they haven’t been fixed before is due to the freerider nightmare meaning there hasn’t been $ to fund the work.
FOSS Synapse has had Rust in it for years now: https://github.com/element-hq/synapse/tree/develop/rust/src
> Your own blog post about the pro version (and the one about forking synapse) basically said that you wouldn't be working on the community version at all anymore?
WHAT?! It’s the opposite! We’re trying to fund the FOSS dev!! Where did it say that?!
> WHAT?! It’s the opposite! We’re trying to fund the FOSS dev!! Where did it say that?!
Ah sorry it looks like I also misunderstood this blog post from 2023: https://matrix.org/blog/2023/11/06/future-of-synapse-dendrit...
I thought the community version was left for the community to maintain when it was forked. Especially because the foundation said they wouldn't be able to fund further development.
But I see now that it just meant that element would become the official host for synapse.
With the other blog post (the recent one about synapse pro) it just goes into so much detail about the shortcomings of the community version that it kinda implies they won't be fixed. After all, if that were the case one could just wait and the problem would solve itself.
But I apologise for drawing the wrong conclusions and arguing for them here.
I do really understand you're in a difficult position. And I don't have a solution either.
i’m terrified now that the various blog posts haven’t been clear enough about the plan though - seems like we should have made it much clearer that FOSS synapse is still the primary project, and everything will happen there first (other than optimisations to support massive scalability)
For what's worth, I stopped promoting Matrix for that exact reason. And instead of waiting for Matrix to sort itself out, I gave XMPP a second look, and ended up adopting it for what Matrix was promising to become "soon". That was 8 years ago and the outcome would be the same today (or maybe more favourable to XMPP as it feels to have evolved more than Matrix in that timespan).
The rugpull can be observed here [2] by the playbook so many others have used:
- Create commercial offering
- Add some breakwards compatibility breaking shit to the repo
- Declare the project dead
- Old users are now basically forced into "the new" product
[1] https://matrix.org/blog/2023/11/06/future-of-synapse-dendrit...
Neither is not getting paid for work/services. If one works hard on things and can't pay their bills, that's unsustainable. Why is developer time worth nothing to other developers? If you can write it and host it, do it. If you can't, and the folks that can charge for it, hand over your credit card so they can continue to do it. Or watch it die when they have to abandon it to make a living writing code for someone that will pay for it I guess.
No one owes us their time and effort for nothing.
It does not seem to me like the only choice is either FOSS or massive enterprise 5-figures/year here - what am I missing?
But I have not asked, that's true.
I feel like though they would have some accessible paid plan for “smaller” users that want to support the development of the project long term - I believe this is the most sustainable way to build complex “big” FOSS projects - apart from having someone sacrifice themselves for the greater good I guess
Which seems to me they want to talk to purchasing departments, not end users. But let's pretend I'm trying to give them money anyway. No Buy/Sign Up link, the Get Started link in the top right and Get In Touch link on each price box lead to the same place: https://try.element.io/get-started?utm_source=pricing-busine...
Again, "consumers go away and use the free tier" or "learn more" about their enterprise offerings. Learn more brings you here: https://element.io/server-suite with a generic sales pitch and a "Get Started" link on the bottom.
Which sends you right back to the second page. So right now, apparently you can't pay Element money
Conduit[1] seems to predate Synapse Pro and I've ran it for a short while. It consumes about 32 MiB of RAM for a server with at most a dozen registered users. However, Conduit, like Dendrite, has fallen behind on implementing the latest revisions of the Matrix API. Thus I cannot fully recommend deploying Conduit unless you are aware of exactly which clients and rooms your users expect to interact with.
(This would at least for me have had a very different taste and reception if the private for-profit alternative would have been done under a different branding than "Synapse Pro" and not hijacking matrix.org community channels to promote it. They don't seem to be able to keep their hats and chairs properly separated)
How? Synapse is still being developed, this is just a commercial addon for huge customers.
> and not hijacking matrix.org community channels to promote it. They don't seem to be able to keep their hats and chairs properly separated
matrix.org has never excluded mentioning commercial entities, past blogs have frequently included information about etke or beeper for example.
Is the Rust version reducing vulnerabilities? It looks like the main reason for the Rust version is reduced CPU use, and Python does not suffer from the memory safety issues that C and C++ have.
Edit: oh ok it's still alive but just not very active. That's ok. I think synapse is fine as long as the long-standing issues would be fixed. Like big federated rooms.
Synapse Pro is some alternative new worker implementations for Synapse designed to let enormous deployments scale rather than run out of headroom. Those happen to be proprietary.
FOSS Synapse is still the same project, and the main “real” project, and is still FOSS, same as it ever was.
If Synapse "Pro" isn't synapse, it should really be called something else.
But if you are merely handling tens of thousands of users, no problem. And if you have the budget to handle millions of users, maybe you have the budget to pay development as well as operations staff?
ejabberd is a good place to start an xmpp chat service. Facebook messenger was built on it, WhatsApp was built on it, Riot Games chat was built on it. I'm sure there's others.
If you attract enough users, you might end up needing to modify it for your needs, but that's a nice problem to have. There's a fair amount of presentations at tech events with bits and pieces of infra knowledge from FB, WA, and I think Riot Games. Although some of that is fairly obsolete... You don't need to work as hard to scale Erlang today, the BEAM is a lot more scalable out of the box in 2025 than it was in 2011, and you can also have a single host with a lot more ram and compute today too.
And I believe the free software movement is lamenting that?
Maybe clear communication would work better? Use words in a meaning, people understand intuitivly?
A "free dogs" sign at a dog shelter would indicate them giving them away for free (they usually don't intentionally). (And still your own cost of transporting etc)
A "free dogs" sign on a protestors slogan outside a dog breeding for food factory, probaly something different.
In context of software, free software creates the expection free of cost mostly. And by now slowly and slowly, also free to do what I want. Sort of. Because there is this ideological battle of what is more freedom, free also to comercialize again, or is more freedom to allow anything? And that battle and different definitions, is pretty confusing to outsiders. The GNU version I don't like for that reason.
"To understand the concept, you should think of “free” as in “free speech,” not as in “free beer.” We sometimes call it “libre software,” borrowing the French or Spanish word for “free” as in freedom, to show we do not mean the software is gratis."
It makes things confusing without need. Result is, outsiders don't help so much or want to get involved.
The whole point is that the phrase "free software" has nothing to do with money.
It almost seems like you want to make things simpler by just merging the two concepts. But that's fundamentally not what the movement is about.
But it does in reality.
Off the top of my head, I can't think of a better alternative. (Libre is nice, but I think most people would probably get that worse, not better)
Obviously nothing is perfect, but this criticism doesn't make sense to me.
(That doesn't mean making beer is free, same as software, so I don't see the need to complicate simple things here)
and which english word would you suggest intuitively communicates the meaning of free as in free to modify and share? there isn't one because the concept itself is unfamiliar to most people.
If you add a free to use, I would agree. For me it has both meanings. And in your definition it is kind of implicated, but in others not necessarily.
" To understand the concept, you should think of “free” as in “free speech,” not as in “free beer.” We sometimes call it “libre software,” borrowing the French or Spanish word for “free” as in freedom, to show we do not mean the software is gratis."
The last part is bothering me. Because yes, de facto it means free as in beer. I don't have to pay money to download the source and run my own version, except paying for my infrastructure.
but it shouldn't, that's one of the problems. developers need to be paid. more and more projects are in trouble because of this expectation that i don't need to pay for libre software.
HN generally does though, and that's why I said it here.
> Maybe clear communication would work better?
It was clear.
Universe scale at Synapse “Event Horizon”
edit: what’s to stop community from re-implementing the worker component in rust and contributing it to the “pro” version?
The essence of Open Core suck, which Matrix has here committed to, is actively discouraging community work related to the bits that differentiate the proprietary version. History says any pull request improving the FOSS version too much, in the wrong direction, will be left unmerged.
This is how "communities" die.
This bait-and-switch leaves a really bad taste for me though; I now feel silly for having advocated for Matrix, and will cease henceforth. You can lament the free-rider problem all you like, but suddenly making it a big deal now betrays that you do not take seriously enough your own promises, and were clueless about the fundamentals all this while. This is not meant to be vindictive (I wish Matrix success as a product; it's certainly useful), but simply making clear that the expectations and standards are different for a "community" -vs- simply a business that sells a product. It seems to me like this disqualifies Matrix from consideration for "public infrastructure" IMHO. It would have been better if the differentiation between the paid and free versions wasn't scaling, but some other features not critical for public use (or provided as a non-free plugin), but something that enterprises would consider useful.
Please don’t conflate Synapse with Matrix. Yes, if a govt has a “we must only spend money on pure open source products” policy then they won’t buy Synapse Pro. Perhaps they’ll end up scaling by running hundreds of separate smaller FOSS instances instead. Perhaps other server implementations will step up as FOSS (and probably promptly be undermined by the same freerider problem).
> It would have been better if the differentiation between the paid and free versions wasn't scaling, but some other features not critical for public use (or provided as a non-free plugin), but something that enterprises would consider useful.
We’ve tried that (https://element.io/server-suite) and it hasn’t been enough.
Will this finally be enough, or will it end up so that even more features are going to be available only on the proprietary version?
I imagine anything else would be quite irresponsible in a geopolitical climate where sanctions could pull the rug out on critical communications infrastructure.
eg: https://element.io/blog/senators-implore-department-of-defen...
If Matrix is just "yet another service (SaaS)" competing with Slack, MS Teams and the like it becomes a much less compelling argument for public infrastructure. You better have a product that is so much superior in functionality that it is worth the switching cost for customers.
I'm not sure I follow the logic here: Governments can self-host their own infrastructure and avoid potential sanctions without it having to be open source.
Meanwhile, empirically, all Governments continue to spend eyewatering amounts of proprietary software - whether that's from Microsoft or Adobe or Google or whoever. It's excruciating that in practice, only a miniscule proportion of that gets routed to open source - and even then, it's typically on a "we'll pay for features, but not maintenance" basis. So I'm not seeing any “we must only spend money on pure open source products” in practice; the closest is "if we're paying for something to be built, it needs to be open source" (which makes perfect sense, imo).
> If Matrix is just "yet another service (SaaS)" competing with Slack, MS Teams and the like it becomes a much less compelling argument for public infrastructure.
Matrix is an open protocol - it's not remotely "yet another SaaS". There's already a selection of different server implementations; some FOSS, some proprietary. You can switch between servers, and switch between clients. The fact that one server has added acceleration to enable successful massive-scale implementations and kept that as proprietary (for now) does not undermine the protocol - or stop govts from purchasing it, imo.
I think we may need to agree to disagree on this one tho :)
Performance has always been one of the most glaring issues with using Matrix. The article seems to be intentionally obfuscating the fact that performance and scalability issues stem from the activity a server participates in rather than just the number of users it has. If you're in a rather large room with hundreds of members and it's constantly active, you'll quickly notice your server struggling to keep up.
As a Matrix user, I feel like I'm being rug-pulled because Element HQ decided to gatekeep their solution to this problem that has plagued the community for more than half a decade.
The reason that small servers struggle today is not because their workers are written mainly in Python rather than Rust. Hell, they typically don’t run workers at all. The slowness comes from state res being slow algorithmically; state storage being slow and inefficient; wasting time trying to talk to dead servers; no support for “thin nodes” but always doing full mesh federation; the fact faster room joins never got finished; etc etc.
ALL of this would get fixed on FOSS Synapse. The point of Synapse Pro is to try to get $ to actually fund that work. Only massive-scalability stuff is in scope for Synapse Pro: all other features, perf optimisations, maintenance, security work etc will land in FOSS Synapse as it has for the last 10 years. Assuming that this gambit works and there’s any $ to fund it, of course.
My voice is mostly coming from the perspective of someone who has tried repeatedly to bring friends and family to Matrix but every time the experience is subpar. It's frustrating because most of the problems they experienced (including performance) have been known about and complained about for a long time.
Maybe the messaging in the article was a little off. The "attack" it makes on FOSS Synapse reads to me like it will never get the fixes it needs to make using Matrix more approachable to new users.
FOSS owes you literally nothing. Go build what you want for yourself, geez. Oh, and feel free to give it away to the world… OR NOT!
There's a lot of moral support and ecosystem gain (eg bridges) from the community. Why bother with that anymore if we're just getting the "broken implementation"?
I think part of the issue here is that they're trying to compete with the enterprise IM solutions. That doesn't really work as the others are way better funded and have benefits that element can't deliver (such as the wide Microsoft M365 + EntraID ecosystem). A Microsoft shop is never ever going to choose this over teams (horrible as the latter is).
You’re not. All work (other than stuff which primarily benefits scalability for enormous deployments) should land on FOSS Synapse - particularly core performance improvements like faster state res. This is just trying to fund it.
And no, this is not us trying to compete with Teams - this is us trying to force big Matrix deployments to actually route $ to Element to fund upstream dev, rather than use System Integrators who win the big tenders and then mysteriously fail to route any $ to us.
But anything that could be considered public infrastructure better be able to scale to millions of users!
I wish you had chosen to gatekeep some other features to differentiate between enterprises (with members numbering in 10^1 -- 10^6) the open version; public infra should ideally be able to handle >>10^6 members.
What did you think was going to happen? That's their profit.
Professional software developers work for the customers that pay them. Doing free work isn't a sustainable businesses model, not for anyone.
If they're not paying you then they're not your customer and you don't owe them anything.
Professionals build products. Don't waste time/money building anything but the best possible product. The best product creates the most value for users and customers pay for that value. This functions like a corporate tax on profit.
Software has zero marginal cost which let's people who aren't generating much if any economic value benefit. It's nice but it's not a business.
Everyone benefits from a healthy ecosystem where developers are paid and build products that end enriching the public domain sooner or later. Align your incentives with your paying customers and you will prosper.
I guess licensing and FOSS don't work together.
But then the pro version doesn't benefit from the open-source scrutiny of the community version, especially in regards to cryptography and security.
What's the reasoning for this technology split?
But support is deprioritised in favour of Synapse (Python).
Moving the goalposts with Synapse does not rejuvenate Synspse, it creates a fragmented system.
Get used to being boring. I am not moving to an ever centralised Synapse only ecosystem - a single implementation is not a federated protocol. That the new Element mobile client only supports Synapse was a huge red flag.
If diversity in implementation is not encouraged with action I'm back to XMPP (+ Jitsi).
This is not true. Element X requires Matrix 2.0, which is currently only in Synapse because it has to land somewhere first. But Dendrite is implementing it (best effort, thanks to an amazing community contrib) too: https://github.com/element-hq/dendrite/pull/3493 and at conduwuit (one of the conduit forks) is too: https://github.com/girlbossceo/conduwuit/pull/666
There is absolutely nothing trying to lock EX to synapse.
A lot of FOSS users conflate freedom with "I get free shit in perpetuity forever" and then whine when they see something that they don't get for free, even if nothing was taken away from them.
Best wishes to the Matrix project.
It's heavier than any other protocol or chat program I know.
I'd really wish to see an official non-Python OSS version though, to make it easier to run on a Raspberry Pi or whatever other tiny homelabs people usually host things on. When I tried Synapse, it essentially ate the whole memory and was constantly making the fan turn on.
You can read it here: https://www.process-one.net/blog/matrix-and-xmpp-thoughts-on...
TL;DR: Matrix protocol is based on document replication and synchronisation, while XMPP is based on message passing. Matrix requires to constantly replicate and merge the data. It is an intended feature, but as it does more, you cannot expect to scale to the same level.
That said, we want ejabberd to be able to suit all types of messaging needs and we have already implemented 1-to-1 gateway with Matrix server inside the protocol. We are fine tuning our chatroom interop with Matrix. It means that we have already most of the code in place to support Matrix chat (as we need to implement the replication part). It means, the next step could be to implement full Matrix support for those who would rather use Matrix clients.
Disclosure: I am ProcessOne Founder, XMPP developer since 1999 and working on ejabberd since 2002.
> tl;dr is more "need to be able to keep paying people to work on this".
https://mastodon.matrix.org/@element/113844400165509238
That's your problem. I only had paid attention to this project in the past because it was under a permissive open source license.
A weird thing has happened while I've been reading this thread where I've noticed some outsized cynical or negative comments and realized they're both your user, then I remembered your bad faith arguments in the thread about the license change. I'm too lazy to check; do you come to every Matrix/Element thread to be uselessly negative or just these two?
Disappointed but not surprised.
In short they are saying that if the nation states hadn't opted to use Matrix they would never have bothered developing this Rust version, which only shows that they were never sincere about developing a performant open source version, not to mention their involvement with some suspect Israeli comms companies and the phone home agendas.
Interesting question - will their "nation state" level customers have the right to inspect the source and make changes to it?
Just waiting to have this post flagged.
At that point, it's nice of you to assume that they are even capable of making it performant. But it is indeed "remarkable" that the people who started this, and who couldn't get anywhere with it, for over a whole decade, are still behind it today.
I have run a Synapse server for almost 8 years, by my reckoning. For most of that time I have been a $15/mo supporter.
For several years, I had been hoping Dendrite would be released, along with a migration path to get my users over there. Synapse's resource usage is not great. I do run workers in order to improve performance.
I'm waiting for an official migration path because I don't want to have to migrate my community again like I did when we moved from Slack to Matrix. It takes a lot of work just to move people over, and you always lose a couple people, which is a serious cost.
Early last year I learned that they had put that on ice due to money issues. So there wasn't much hope of moving to a lower-cost Matrix implementation without a lot of headache. This makes sense. Building a homeserver implementation while maintaining an older one is expensive.
For a year or more we've had quite a few blog posts saying that there's not enough money and that large organizations join the Matrix Foundation. This makes sense. Those organizations have enough money to keep it going, unlike my small monthly donation, which doesn't really matter all that much in the grand scheme of things.
It's been quite a while since we've seen a new user-facing feature, and longer since we've had a selling point (which I could use to answer the question "why should I move to Matrix?"). It makes sense to prioritize functionality over new features, particularly when you've got a limited budget. But we still don't have some features which are very popular in Discord and Slack, like custom react images; these are implemented in other clients, like Cinny, but not Element.
Last year they released Sliding Sync in Synapse, deprecating the Sliding Sync Proxy which I had been running to support clients who wanted to use Element X (a new client implementation). I personally haven't switched since Element X does not support Spaces. Moving Sliding Sync into Synapse saved me some resources supporting those clients. It was a little hard to tell when it was safe to remove the Sliding Sync Proxy; I had to track a couple Github issues. Matrix used to have a public roadmap, but it's no longer updated, so it's hard to keep up with the status of different features in development.
After that they released Matrix Authentication Service (MAS), which is an additional service to deploy that moves the internal authentication functionality out of Synapse and interfaces with Synapse using OIDC. I haven't deployed it yet. They say it will eventually get rolled into Synapse, so I'm intending to wait for that.
All in all, it does not feel like the things I want, and (assuming I'm not a completely unusual case) that the community wants, are high priority for the Element team. Donations the size of mine don't make a difference for their budget. They're spending what budget they do have on refactors like MAS, which don't seem to impact usability (though perhaps they do if you have a massive homeserver). They spend time and effort supporting new features which make Element X faster (Sliding Sync) but have not yet implemented all the core functionality (Spaces) so there's not much reason to move.
I concluded a few months ago that our interests are not aligned any more and stopped donating. I know I'm not owed anything for my donations. I donated to support a project which I was excited about. This announcement, and that RAM graph which I will never see on my own server, makes me confident in discontinuing my support.
I do not feel like Matrix/Element values its community any more.
They broke out synapse's authentication into a separate service, only to plan to roll it back into synapse later? There's probably more to it than that, right?
Therefore, we needed a basic Matrix-aware OIDC identity provider which could ship with Synapse and other homeservers in order to auth users (and optionally delegate to fully fledged IdPs like Keycloak). So we wrote matrix-authentication-service (MAS) in Rust, which is released as FOSS: https://github.com/element-hq/matrix-authentication-service.
This is released as a separate project because: a) it's written by a different team, b) it's intended to power OIDC auth for other servers than Synapse. For instance, https://github.com/element-hq/dendrite/pull/3493 is the pull to make Dendrite support OIDC via MAS.
In future, we may end up for admin convenience also bundle it by default inside FOSS Synapse (especially given Synapse is part-Rust these days) - so that folks running `pip install matrix-synapse` magically get OIDC-based auth without having to also run a separate MAS. However, this is a while off, and even then we'll continue to support MAS as a standalone service as the primary configuration.
(N.B. none of this has anything to do with Synapse Pro, and is an example of Element continuing to pour the majority its effort into FOSS work like MAS and native OIDC in Matrix...)
> It's been quite a while since we've seen a new user-facing feature
I'm not sure this is true, as you say yourself in the following paragraphs. On the Element side, we shipped Element X as the much-needed rewrite of the old Element apps. On the Matrix side, Matrix 2.0 is a step change in terms of instant sync/login/launch, QR login, native-OIDC, invisible encryption, etc. From an end-user perspective, my experience of Matrix has improved unrecognisably in the last year. I can see how this might feel different if you're stuck on the old stack.
> All in all, it does not feel like the things I want, and (assuming I'm not a completely unusual case) that the community wants, are high priority for the Element team
We've tried to prioritise usability, stability and performance over everything - including features. I'm sorry if this doesn't match your needs though.
> They're spending what budget they do have on refactors like MAS, which don't seem to impact usability.
Some of the usability improvements that MAS provides are:
* 2FA and MFA (which is frankly embarrassing that we didn't have before)
* QR login (for parity with Discord/WhatsApp/Signal "just scan this QR code to login")
* Refresh tokens, so that leaked access tokens expire rapidly
* Integration with password managers
* Password reset flows which work on all clients (e.g. password reset doesn't exist on the legacy Element mobile apps)
* Huge security improvements by actually following a mature auth standard (OIDC) rather than making up our own one. We've had some near misses on the old auth system for failure modes which OIDC fixed years ago.
> They spend time and effort supporting new features which make Element X faster (Sliding Sync) but have not yet implemented all the core functionality (Spaces) so there's not much reason to move.
Spaces is already there on the SchildiChat Next fork of Element X Android. We'll add it to Element X Android later in the year - but it's been delayed as the paying customers tend not to care that much about spaces (they just want a WhatsApp clone).
> This announcement, and that RAM graph which I will never see on my own server, makes me confident in discontinuing my support.
The hope is that the $ from Synapse Pro allows us to fund FOSS Synapse dev to improve the core perf & resource usage of FOSS Synapse - which in turn will reduce the RAM usage for everyone. (But massive-scale scalability will remain paywalled for now.)
To be clear: the reason small Synapse servers are slow is NOTHING to do with the workers being written in Python rather than Rust. We will use $ from Synapse Pro to speed up Synapse (and Matrix, at the protocol level) for everyone.
> I do not feel like Matrix/Element values its community any more.
Sorry. If we didn't value the community I wouldn't be sitting on HN trying to explain why we've done this. Thanks again for supporting over the years.
- They own Element, the by far most used client
- They own Synapse, the by far most used server
- They own dendrite, the only matrix server so far that even comes close to spec adherance with synapse, which has been strangled (as evidenced by the last release almost half a year ago [3])
Because everyone is running synapse, everyone needs to keep running synapse or risk losing compatibility with the federation.
This is not what the sales brochure for matrix read like. Like, at all.
Do y'all believe it's random chance that the official synapse docker-compose does not come with async workers? [1]
And now check out the terrible setup process for setting up the whole "workers magic" (which, mind you, is unavoidable the moment you have some bigger rooms joined) [2] - Like who on this godforsaken planet considers this valid even for 2022? Pair that with how easy to handle the base documentation is and things start to smell. They have the stench of trying to artificially create a moot by making it "too hard for non-pros".
I understand synapse and dendrite development need funding, but this is not the way to go. All good will towards Element as a steward of matrix I had is completely eroded by now and I assume that will be the case for many others too. Forks are bound to happen at this point and I'm looking forward to the ecosystem become more diverse again.
Oh, and did you know the Matrix Foundation is infact a UK based Inc. controlled by 5 people in 5 companies, of which 4 companies are residing at the same address [4] and the fifth has an odd interest in NFTs and other BS?
[1] https://github.com/matrix-org/synapse/blob/develop/contrib/d...
[2] https://matrix-org.github.io/synapse/latest/workers.html - note how this page is already marked as deprecated. Prepare for it to vanish altogether soon. You are not supposed to set up workers, you are supposed to beg element to run a decent matrix server
[3] https://github.com/matrix-org/dendrite
[4] https://find-and-update.company-information.service.gov.uk/c...
You sound like you are getting Element mixed up with Matrix. Element releases almost all of its implementations as FOSS, and to help Matrix grow, will make those as good as possible for average server sizes. It’s only gigantic servers which actually require the new proprietary modules. Given everyone who uses FOSS Synapse can keep doing so, and given FOSS Synapse dev should hopefully improve and accelerate if actually funded by $ from Synapse Pro, it’s not fair to say that “FOSS is essentially a marketing tool for Element”