The Matrix.org 2018 year in review | Hacker News Reader

The Matrix.org 2018 year in review

matrix.org

113 pointsArathorn37 comments

This is a massive post (I didn’t have time to write a short one so I wrote a long one), but there’s some vital stuff buried in it for those interested in Matrix, including:

* Details for testing the redesign of Riot (https://riot.im/experimental)

* A teaser of fan-out routing

* The potential roadmap options for 2019

* The endgame for E2E encryption, Matrix 1.0 and a whole lot more.

Any questions welcome :)

For some reason changing color theme isn't working in the experimental client (Firefox 65.0b6 / Linux).

that’s because the redesigned skin doesn’t (yet) implement tinting - it simply doesn’t have the same opinionated accent colour (default green) of legacy Riot.

Are there any plans for adding new transport protocols (for c2s and s2s communication other than HTTP)?

I didn't see Fractal being mentioned. IIRX they have received some founding from Puri.sm recently. Sorry if I messed it as I didn't read the entire article.

The article just discusses what the core team has been up to - it deliberately skips the wider community (for better or worse).

The youtube video at the end shows an alternative C2S and S2S transport being used - similar to the one described at https://fosdem.org/2019/schedule/event/matrix/. Atm it's experimental and only works in closed networks and isn't yet released, but the plan is to add it as an official alt transport once we've got it working nicely in open networks. It's about 30-50x more bandwidth efficient than the default HTTP/1.1+TLS+JSON transport.

I'm glad to see there is finally a binary C2S transport in the works.

Are you planning to reduce the number of brand names that you operate ?

Vector.im vs Riot.im vs Modular.im ? The FAQ of modular.im asks you to accept a TOS from riot.im . The website demo is on riot.im, but the hosted product is modular.im .

And when you write in to modular.im support, you get a reply from an @matrix.org person. Jobs are applied for on vector.im ...however riot.im twitter page says "formerly vector.im". Modular.im (your hosted slack alternative) page has NO feature list. People are expected to go to riot.im to understand features and play with the demo. In fact your hosted product modular.im has no links to even download the mobile/desktop clients.

People get surprised when I tell them about riot/matrix having a hosted slack alternative that is significantly cheaper and is open-source+E2E. I believe one of the reasons is that have 4 websites rather than one.

Not sure why this situation exists. It doesn't help when making a case for it in the community or company. Gitlab does this well. Please fix.

I don't think you should be uncomfortable with the idea of mixing your open-source products with closed source code (for e.g. modular.im migration tools are closed source) - confusion is the bigger problem.

You made the right choice in focusing on Synapse and letting Dendrite stall. You can build one reference implementation well and let others do as they please. You should do the same on the desktop client.

Gitlab is a single company who makes a single product. Matrix is a protocol with a whole ecosystem. You simply can’t compare them like for like - a better comparison would be Matrix and Git... where nobody is surprised that a multitude of different companies exist within the Git ecosystem (Gitlab, Github, Gitkraken, etc) - some of which are multiple products with different names built by the same company (Github and VS Code for instance), and some of which have a different corporate name.

For Matrix, the naming is actually pretty simple:

Matrix is a protocol curated by the Matrix.org Foundation.

Riot is a matrix client (which is just one of many in the Matrix ecosystem, hence not calling it “Matrix client” for the same reason that Firefox isn’t called “Web browser”.

Modular is a matrix hosting service, which can support any Matrix client, and isn’t tied specifically to Riot, and should be one of many in future, hence not calling it “Matrix Hosting Services”, just as AWS don’t call themselves “Web Services”.

As it happens, Riot and Modular are made by a startup called New Vector, which also hires most of the core Matrix team. We don’t expect anyone to know the name New Vector though, any more that we expect people to know that Slack’s legal entity was called Tiny Speck for years. It generally only comes up when hiring.

So: yes, it’s surprising that there are 4 brands flying around here - and many many more if you consider the wider Matrix ecosystem of clients and servers etc. But it is completely inevitable. This is the primary difference between Matrix and (say) Rocket.Chat or Mattermost or Slack or Discord. The power and flexibility of an open ecosystem comes at the expense of needing to give the different parts different names, just like the Web does.

If you look at Gitlab, the intent is similar to yours and is no different. Just that the brand is managed much better.

For example, Runner is an open source project that is used to run CI jobs. https://docs.gitlab.com/runner/ . Omnibus is packaging tool (https://docs.gitlab.com/omnibus/README.html) and Gitlab is the git tool itself. If you peek at the top right corner of the documentation site, you will see these three big headings.

Here are your gitlab clients - https://about.gitlab.com/partners/ (including all the native apps)

Gitlab CE is the open source software. This is similar to you having riot. Gitlab.com is the hosting service - similar to you having modular. There is no difference in intent or licensing... im talking about brand. For the end-user, the benefit in incalculable - one website (and Google search keyword) to find out all the options and all the information. I send people to Gitlab.com if they want to run it themselves OR if they want to buy the hosting service. In fact, Gitlab had org vs com domains for open source vs paid software .. when they then merged (https://about.gitlab.com/2014/03/07/moved-to-dot-com/)

There is no reason for you to identify Vector.im as separate from Riot.im as separate from Modular.im . For me as an end-user, it doesnt matter - Riot.im could have a link to download and a link to host and a link for about us.

It is not about inevitability - you are outwardly branding stuff separately that is not needed and can be unified under a single umbrella to reduce confusion.

The intent is different. Gitlab wants to distribute and sell one unified thing, Matrix wants it to be clear the pieces are just that: pieces you can interchange with others. It's not important to how Gitlab represents itself to the outside that you maybe could replace Omnibus or the Runner with other software, for Matrix it's important.

There is only one piece of software that is componentized and named here - riot.im which is a client. Your argument is a straw man argument because I don't see websites for Synapse or Dendrite..which is their core server. That is still part of matrix.org .

Vector is the name of their company and modular is the name of their hosting service. So the argument about representation doesn't stand. The brand theory still holds.

Just going through your FAQ...

>"Why are you called Matrix?"

---

>"We are called Matrix because we provide a structure in which all communication can be matrixed together."

>"No, it’s nothing to do with the film (although you could go and build virtual worlds on top of Matrix if you wanted :)"

Am kind of surprised there is no mention of William Gibson there.

Honestly Neuromancer didn’t factor explicitly in picking the name, other than through ambient connotations. Or for that matter Doctor Who, which used the name to describe the Timelord’s computing system on Gallifrey :)

I had forgotten the Dr Who reference, though I only dip into Dr Who now and again. I had always assumed that the title of the film 'The Matrix' was in homage to Neuromancer. I had never considered that Gibson might be doing a homage to Dr Who.

Matrix is a really exciting project. It's really great that an open and decentralize alternative has a viable shot to mainstream adoption for this generation of communication platforms -- something that never materialized in Facebook's rise to dominance.

One thing about this roadmap post that is confusing to me is that the mobile client work, specifically iOS, seems under-prioritized. I'm assuming the focus on UX the last half of the year has been intended to put matrix in a position to compete with the mainstream apps for adoption, so having solid mobile UX seems just as important. Is there a reason matrix expects users to be desktop-centric or that the mobile UX is already good enough to compete with Slack, Discord, etc?

The main reason for mobile being underprioritised is that many of the mobile devs (who are based in Rennes in France) are doubletiming with helping the French government project and so are quite stretched for time.

Also, for better or worse, we have entirely separate codebases currently on Web, iOS and Android. The reason we do this is to ensure that other app developers can use the same proper native matrix SDK in their apps rather than be forced to use a device abstraction approach like React Native. However, it means that typically the web SDK leads the way (as it’s the easiest test bed for new features, and several of the core spec team work on Riot/Web, so new stuff happens there). However, we’re starting to see the balance shift: the new Android SDK and app is more advanced than the JS SDK, and we might even experiment with using Kotlin Native as a way to share that code to other platforms.

Eitherway: we’re aware that mobile needs more love and investment and we’re trying to fix it :)

> ensure that other app developers can use the same proper native matrix SDK in their apps rather than be forced to use a device abstraction approach like React Native

Thank you!

:) just hope it's worth the tripled effort of us having to build and maintain matrix-{react,js}-sdk, matrix-ios-sdk and matrix-android-sdk versus writing once and running everywhere... ;)

That makes sense, thanks for the additional info!

Unless Apple drags their feet with PWA implementation a native iOS app might not be needed?

chat clients are surprisingly heavy, and i suspect tha PWAs will never perform as well as a handcrafted native app. But we shall see!

What this post doesn’t mention is the failure to tackle a fairly significant problem[0] leading to Matrix bridges being banned from at least one major irc network (hackint) and a number of irc channels elsewhere. If you were thinking of using Matrix because you thought it might allow you to access all your old chat systems as well, think again.

[0] The Matrix bridge model fundamentally doesn’t map to how IRC works in reality, and causes massive headaches for IRC network and channel operators.

Matrix works great with Freenode. What exactly is the problem? How does it not map? I authenticated with my freenode nick/password, and it's acting as a regular bouncer pretty much…

There’s a number of larger channels which have banned Matrix users, and various smaller channels I’m in have been considering banning them due to the privacy concerns of Matrix.org (based in the U.K., iirc, and potentially subject to an easy court order requiring them to release their data) keeping a full, eternal log of all communications in any irc channel where a Matrix user has joined.

Freenode is also not the entirety of IRC.

The main reason why some channels have banned Matrix in the past was instability in the bridge causing occasional netsplit style behaviour where all the Matrix users would bulk-part and bulk-join. Since stability was fixed earlier this year, it should no longer be a concern.

In terms of privacy concerns: yes, Matrix.org is currently provided by a UK legal entity, just as many IRC servers are too. But if you want privacy from a malicious server admin or government or whoever then you should be using end-to-end encryption, and you shouldn't be using IRC. I'm not sure the legal jurisdiction makes much difference (unless the UK outlaws E2E, which then becomes a whole new can of worms).

If you're worried about the fact that Matrix rooms contain scrollback, then it's (again) unfortunate that nobody has cared enough about this to file a bug on https://github.com/matrix-org/matrix-appservice-irc/issues to request configurable retention per bridge. Meanwhile, https://github.com/matrix-org/matrix-doc/issues/447 i s the general spec issue if you'd like to upvote or comment on it. But in the end, Matrix is FOSS, and we kinda hope that folks will contribute stuff like this that we simply haven't got to yet.

It is, in general, not a concern of IRC users that Matrix is broken past what they need to do to protect themselves from it, and attempting to fob Matrix development off on them is really not on.

While I agree that many communications should go over e2e chat, there are a number of low-real-risk channels on IRC that occasionally discuss drugs and similar, and adding the additional risk vector that ten years down the line some prosecutor decides to dig the logs up through Matrix.org when everyone else has long since lost them isn’t really cool. My understanding from last time I went through the codebase is that logs will not be deleted even if all Matrix users leave the room.

> attempting to fob Matrix development off on them is really not on.

My point is that if you have requirements for a bridge (eg configurable scrollback retention), you should submit a feature req to let us know about it at the time rather than ragequitting and telling people that matrix is broken.

Anyway, point taken that we need to make scrollback retention configurable (whether for bridging or in general); have bumped it up the todo list.

Oh, right, the log "issue". I think it's extremely naive to try to keep an "unlogged" IRC channel, since any user could secretly "leak" the logs.

So it's true that the blog post doesn't enumerate each and every bug in Matrix today.

You're right that there's a fundamental difference between IRC and Matrix. In Matrix, the room history and state is represented by a single datastructure replicated across the participating servers. In IRC, each user effectively sees their own version of the room history - for instance the only rule for whether you can see messages in a room is whether the server considers your client joined to that room (given lack of scrollback in IRC). Similarly, poweruser IRC features such as the ability for chanops to send messages in a channel which are only visible to specific users are hard to map to Matrix's model. Matrix expects it to be consistent and well-defined as to who is in a room at a given time, and who can see what message.

I think the actual bug you're talking about is https://github.com/matrix-org/matrix-appservice-irc/issues/2..., which boils down to the fact there is a race when the bridge starts up during which messages are synced between Matrix and IRC without yet knowing whether the Matrix participants in the room have yet joined the IRC side of the bridge. Now, in practice this hasn't been a major problem because: a) the bridge starts up fast, b) nowadays the bridges are stable and are rarely restarted, c) the chances of the membership changing on the IRC side whilst the bridge restarts and racing with the Matrix-side membership are fairly slim.

Possible fixes include:

* Making the bridge refuse to join traffic until it's fully started up. This ends up being a usability v. privacy tradeoff, but would be trivial to implement.

* Implementing per-msg ACLs in Matrix, so that different users in a room get to see precisely the messages which correspond to the points where their IRC client was connected to the network.

* Bridging each IRC channel into a separate Matrix room (horrid, because it means the Matrix users would only be able to interact with each other by wormholing via IRC).

Now, as you can tell from the comment history on the bug, it's simply not got to the top of the todo list to address. It occasionally comes up in conversation with Freenode, but given that the bridge has been pretty much solid for the last 6 months, it's not been a big consideration.

We had NO IDEA that this was a problem for Hackint, and were shocked at https://hackint.org/#20181028_Matrix_Bridging_Sunset given nobody (as far as we know) had discussed the issue with us until after they'd already made a decision. I am also unconvinced that this creates 'massive headaches for IRC network and chanops', and I wonder if historical perf issues with Synapse or similar were more of a contributing factor here.

The biggest issues we've actually seen on IRC bridging are a) handling the floods of spam coming in from IRC; and b) the risk of some twit using Matrix to bridge together disparate rooms on different networks. Both of these are effectively social/reputation problems that we're working on.

TL;DR: if you're having problems with a particular bit of Matrix, please file a bug or upvote/comment an existing bug, and if necessary yell at us until it's fixed. We're unlikely to fix things which we don't realise are problems for people.

[edited for typo & clarity]

Still excited about matrix and riot and similar...

Still need to have methods for moderators and admins to have permissions to view details about other people in the chat rooms.

Things like ip address, server they are federated from(?), client used, etc.

Having run public chat rooms for more than a decade, I've found the need to give some people moderator powers to inspect suspicious users, log info, and ban via subnets and cidr being very important to lessen the flow of many issues.

It only takes one troll to ruin your chat room, and that troll can connect from dozens of ips from dozens of vpns in little time.

The more info we can provide to a few trusted people, the easier it is to detect these things and ban people / networks.

I know this is not a magic pill to fix all the problems with all the people. A combo method with things like having regular users move to a room that takes a certain level to join, and only gaining access to those levels if you've been a known member for x months for example can make a big difference in situations like that as well.

Other moderation tools would be nice too, but without the ip info and such the others are moot in my experiences.

We do have this API already actually: https://github.com/matrix-org/synapse/blob/master/docs/admin..., but it needs UI hooked up to it (and it should probably be in the spec rather than synapse-specific). Agreed that we need more moderation toolsnin general though; we’re working on it.

So far the biggest problem in my opinion is lack of native clients (and servers). The official client is Electron-based, and official server is written in Python. It might be good for prototyping, but hardly convenient for real users due to speed and memory constraints. I hope they will focus more on Dendrite[1] (server in Go) and Nheko[2] (client in Qt) instead of wasting time with Electron.

[1] https://github.com/matrix-org/dendrite

[2] https://matrix.org/docs/projects/client/nheko.html

Nheko is not currently maintained. See Fractal https://gitlab.gnome.org/GNOME/fractal.

See? They didn't even update the information on their site regarding this. Thanks for the link!

That is unfortunately a GNOME application, meaning it uses the Gtk and follows GNOME's HIG thus looking extremely out of place on Windows and MacOS and somewhat out of place on other Linux desktops (assuming Gtk3+ is themed to match the rest of the system, which doesn't help it fit very well anyway).

While I'm sure it's a great option for people using GNOME-based desktops, I don't think it'd be a good idea to bet on Fractal as the Matrix native client, I don't think its authors see it that way either.

Is there a working matrix-IRC bridge for the other way around than usual - to use an IRC client with Matrix?

there have been three attempts at this: PTO, matrix-ircd and most recently https://github.com/nilsding/AgentSmith.

PTO is abandoned; matrix-ircd (https://github.com/matrix-org/matrix-ircd) works but needs contributions and polish to be useful; AgentSmith is brand new and looks promising but haven’t tried it.

Reply on news.ycombinator.com