ara4n

Yup, agreed. We're planning to provide domain vhosting (ie point the SRV for your domain to matrix.org and you'll get your own server instance) which should help a bit, although doing that for free could start to get uneconomic. We also need to work out a nice way to let users migrate between providers.

We're still alive and going strong - currently supporting glossy client development like Vector.im and building out bridges to as many other networks as possible (IRC, Slack, HipChat etc) using http://github.com/matrix-org/matrix-appservice-bridge and similar. In terms of being picked up by a large user base... yes, we need it if we are to be really relevant. And that means relying on folks like those commenting on this thread to try using it, contributing to it, and help us make it better.

[Matrix lead here] The open source clients for both Matrix and XMPP are improving a lot currently. On the Matrix side there's vector.im; on XMPP there's Kaiwa and Conversations.im. If you want to help break the fragmentation and have an open standards based approach please run the clients and help us make them better - it is open source, after all!

Whilst we're not trying to replace XMPP, Matrix.org does provide an entirely different architecture (decentralised and evetually-consistent history, high baseline feature set, HTTP-based, etc) that can be used for group chat/VoIP/etc.

Matrix.org is heavily inspired by Wave, albeit using HTTP rather than XMPp, for those searching a more alive option :)

This is precisely the problem we're working on with Matrix.org - providing a standard API that can be used to bridge together all of these different protocols in one decentralised model. It's better than Trillian in that the defragmentation happens serverside and you can use any compatible client with it (or one of the existing services if you prefer). For instance, we turned on our first Matrix<->Slack bridge this week - see https://github.com/matrix-org/matrix-appservice-bridge/blob/... for how easy it was.

For now, it's an implementation detail (rather than specced) as to how homeservers persist their state. The synapse implementation stores all data unencrypted in sqlite or postgres - /however/ that data may be end-to-end encrypted (we are releasing our e2e support over the course of this week - the first bit of the puzzle can be seen at http://github.com/matrix-org/olm). We should probably store all data AESed in the db to avoid casual snooping too.

This doesn't obfuscate metadata like room membership or profile data however; but fixing this is Hard. For now it's just a fact of life that Matrix servers have visibility on communication metadata - i.e. the identities of who talks to who, and when, and with what kind of event. In future we may support better privacy preserving semantics by evolving the federation architecture: eg running homeservers on clients and using Pond-style hidden Tor services for message transport, or layering on GNUnet as a transport. We've tried to design Matrix to support this sort of evolution, but right now today Matrix provides the same level of metadata privacy ss (say) an IMAP or SMTP server.

There's no harm in having loads of options - you just get a Darwinian survival of the fittest for which app works best. Competition is healthy.

The thing that sucks is that as end users we end up having all our chats and identity fragmented over all these different silos - be they selfhosted ones or proprietary SaaS. There's no way I'll rely on MatterMost or any of the above unless I can access my existing communities (be they on IRC, XMPP, Slack, HipChat or whatever); adding yet more fragmentation into the mix helps nobody.

This is why it's vital to have an open standard for decentralising the conversations between all these different islands that kills fragmentation whenever a new one pops up. And it's actually beneficial to new contenders like MatterMost as it could help them onboard users into their UX and app without having to start new conversations and contacts.

[Disclaimer: Matrix.org is such a standard, providing an open HTTP API for decentralised chatrooms, and I work on it.]

The client-server API of Matrix only uses http+json as the lowest common denominator baseline for compatibility. Folks are more than welcome to implement custom more efficient transports like COAP/CBOR or whatever.

The server-server API is currently just targetting https+json just for expedience in getting started, but nothing to stop us negotiating more efficient transports in future - capn proto has come up a lot in conversation as a possible option. Handling signing is a bit more fun as we currently rely on signing canonical json, but surmountable.

As for binary transfers... well, random blobs are just handled as pure HTTP with a mimetype currently :)

This would be a really cool use for Matrix - the eventual consistency and offline operation stuff is a perfect fit :D

You don't need an email address to sign up currently - the way it works is that it's up to the homeserver to decide how to validate new users (if at all). So Matrix.org uses a CAPTCHA to check you're human, but otherwise just sends a username & password. We also let you optionally specify a mail address, mainly just to prove that we do support validating 3rd party IDs like email (and in future MSISDNs and other IDs).

For gatewaying, the main thing we've been missing is an Application Service (AS) API on Matrix which lets your gateway masquerade multiple users & rooms/channels from the remote protocol. Currently you can write a back-to-back bot which creates, per user, an IRC user on the IRC network and a Matrix user on the Matrix network and bridges them together - this is how the current Matrix<->IRC bridge works. This sucks if you want to easily project hundreds of users & channels from IRC into Matrix and vice versa, though - you need better server support, a bit like IRC Services' architecture.

The good news is that this is very nearly implemented - we got our first AS up and running a few hours ago (it currently just logs traffic rather than bridging it anywhere). It's all happening on the application-services branch of https://github.com/matrix-org/synapse and the doc is at https://github.com/matrix-org/matrix-doc/blob/application-se... and https://github.com/matrix-org/matrix-doc/blob/as-http-api/dr... if you're interested.

Twisted is probably overkill for many ASes, but it could be a great way to do the XMPP & IRC bridging in future.

There is no link between RedMatrix and Matrix.org - the fact that both have Matrix in the name is total coincidence. On the Matrix.org side, we came across RedMatrix for the first time a few months ago when stumbling across http://lists.w3.org/Archives/Public/public-fedsocweb/2013May... in Google, which then linked from Friendica/Red to RedMatrix.

I've never played with Friendica/Red/RedMatrix/Zot, but from what I can see, it's more about federated blogging/social networking than messaging or pure JSON synchronisation like Matrix. Architecturally I'm not sure how they compare - I only just found https://github.com/friendica/red/wiki/zot; we'll have a read and see what the difference is.

Something we're very keen to do with Matrix.org is to build as many gateways and bridges through to other comms ecosystems as possible - so the fact that both Matrix.org & RedMatrix (& Diaspora & Identi.ca & all the others) are operating in a similar space isn't at all a problem - so long as they all talk to each other. It'd be a bit embarrassing if we were building federated communication systems which couldn't gateway to each other!

Yes, it's deliberately much harder for this failure mode to happen:

1. Matrix's baseline featureset is much more comprehensive than XMPP and doesn't yet support API extensions; only datatype extensions. So for a Google to become the defacto implementation and then start lobotomising the featureset they really would not be speaking Matrix at all any more.

2. Federation is completely fundamental to Matrix. All rooms are distributed over all participating servers with no single points of control or failure. So rooms and the network will always live on in other servers, even if a big player becomes a default provider.

XMPP and Matrix are fundamentally different things. XMPP is all about message passing. Matrix is all about state synchronisation. In many ways Matrix is more like an eventually consistent distributed DB like Cassandra or Riak, albeit optimised for storing persistent messages and with open federation (anyone can spin up a node and join the DB) and an HTTP/JSON API.

So: if you want decentralised message history as a first class citizen, use Matrix. It's not just for group chat; it's for any data.

If you want lower-latency message passing with history as an optional extra, use XMPP.

In terms of how to make XMPP cool again - projects like XMPP-FTW and Buddycloud and FMUC are pretty cool :)

...where "We" is the Matrix team :)

Matrix is basically an eventually consistent object database with open federation and pubsub. It's optimised for messaging at the moment - you can use it for group chat, or WebRTC signalling, or M2M/IOT stuff, or anywhere else you want to pubsub data and keep a history. The similarity with Tox is that Matrix can implement a chat system that looks like Tox. The architectures are completely different though: Tox is a big distributed hashtable smeared over lots of peers; Matrix is client/server with synchronisation between te servers. (disclaimer: Matrix is my fault)

We've also been looking at Axolotl as a better-than-OTR for the E2E crypto. But nothing is set in stone yet.

Because we don't want client implementers to be forced to have to jump through end-to-end crypto hoops if they don't want to. The simplest way to send a message in matrix is:

curl -XPOST -d '{"msgtype":"m.text", "body":"hello world"}' "https://matrix.wherever.com/_matrix/client/api/v1/rooms/$roo...

...and we'd like to keep it that way. But you can always insist on only ever communicating with folks who are on E2E crypto clients if you so desire.

It's a shame that XMPP didn't save us from this situation. My hunch is that the baseline featureset over federation was too low: no federated medsage history; MUCs are single point of failures.

We're trying to fix this with Matrix.org - folks frustrated with yet another communication silo might want to check it out and help us tear down the walls between these gardens. (obvious disclaimer: i help run matrix.org)

Oops - I should have called it out more clearly in the blog.

The biggest difference we have over XMPP is probably that messages in Matrix get synchronised over all the participants of a conversation... so you get distributed chat history for free, and no single points of failure on group chat (as you do with XMPP MUCs). And obviously Matrix is plain HTTP+JSON rather than messing around with XML.

One of the reasons we built Matrix is because, in practice, pretty much all the current big players started off using XMPP for developing their chat solutions: Google Talk was originally XMPP; I believe Facebook Messenger was built out at first on ejabberd; WhatsApp was originally XMPP etc; even APNS was originally XMPP!

But ALL of them have ended up mutating it to a proprietary closed standard - and nobody has even tried open federation other than Google's misadventure with Talk. So, unfortunately, it seems XMPP hasn't ended up being the interoperable web-for-IM that we all hoped.

Now, I have absolutely no idea if Matrix will be more successful in solving the problem; the hope is that by keeping it simple and using HTTP APIs there's more of a chance that players of all sizes will start exposing Matrix APIs for federation. Only time will tell. It's also worth noting that end-to-end encryption is a relatively recent potential obstacle: given iMessage and Telegram etc are all end-to-end encrypted, for them to ever federate with Matrix we'll need to support the same semantics and crypto. Hopefully we're going about this the right way (although end-to-end crypto isn't formally specified or implemented yet), but will be an interesting challenge. And one that XMPP hasn't tried to solve at all, as far as I know.

(disclaimer: Matrix is my fault)

SIP+SIMPLE/MRSP's overheads are just as bad as HTTP/1.x, if not worse. We decided to build Matrix after 10 years of fun doing commercial SIP and XMPP work; we have some experience. For instance, the worst scenario we've seen in a real-life SIP/MSRP user-agent in the wild was 50KB of SIP/MSRP nego to send the word "Heh" between two contacts (with a delivery report) :) Meanwhile SPDY and HTTP/2.x improve the overhead situation and pipelining situation enormously - although frankly we haven't seen any real-life problems using HTTP/1.1 yet at all...

The "simplicity" of HTTP I mentioned in the Matrix blurb refers to the fact that RFC2616 is relatively compact and self-contained, whereas SIP/SIMPLE/MSRP involves a huge number of RFCs, different protocols (SIP,SDP,MSRP...) and really is a lot more complicated to implement than just doing GETs and PUTs.

Now, the irony is that rather than being "designed by someone who doesn't know much more than how to build web apps" - it's more the other way round. Our experience is mainly with SIP/RTP/STUN/ICE/TURN etc; genuinely RESTful APIs are relatively new territory for us.

I would genuinely love to know what aspects of the Matrix client-server (or server-server) API is 'not anything like RESTful' - it's not too late for us to change the APIs (they've already been rewritten several times, oscillating between more or less RESTful purity), and half the point of releasing Matrix in its current early proto-form is to get detailed feedback from folks on whether we've Got It All Wrong :)

(disclaimer: Matrix is all my fault)

So, for context: we ended up writing Matrix after literally 10 years of building SIP infrastructure, so we have some experience in SIP (and XMPP's) shortcomings :) The short answer to "why not extend SIP" is that SIP stacks are fairly fiddly to write and get right; and SIP/SIMPLE/MSRP is typically a lot more verbose and convoluted than HTTP (and hugely more verbose than SPDY and HTTP/2.0). There's no support in SIP for distributed storage of state, and there's no support for paginating and storing conversation history at all. In fact, some of the SIP community has even ended up bolting on IMAP(!) for storing conversation and call history :S Meanwhile, given HTTP is pretty simple and completely ubiquitous, so why bother with an alternative stack at all?

In terms of "jesus, not another protocol I have to support" - I completely sympathise, hence quoting http://xkcd.com/927/ in my blog post... We didn't do this lightly :)

One way of thinking of this is that right now you have LOADS of HTTP APIs out there for messaging - Twitter, FB, http://api.ihackernews.com, Reddit... they're sufficiently easy to implement that every new site goes and builds a new one, and developers have to go and learn each new one every time. Meanwhile, none of them will ever federate or interoperate - the whole thing's totally fragmented.

So whilst Matrix is indeed yet another API to support - hopefully it's one that might actually help solve this problem... so that the next time that someone wants to add a messaging HTTP API to their site, they can follow an interoperable pattern rather than reinvent the wheel again. Perhaps a better way of describing Matrix is "two-way RSS for arbitrary data, with distributed history"

Does this help justify things at all? (Disclaimer: Matrix is all my fault)

rpdillon is totally right - the term is overloaded. We're using "realtime" exclusively here to mean synchronous human communication - like the RT in WebRTC. In other words, IM or VoIP - anything where you expect to get a response from the other user inmediately rather than waiting asynchronously.

This of course clashes with the RTOS style use of the word, and we aren't claiming that Matrix has any hard realtime timing guarantees at all, other than the soft realtime behaviour media stacks like WebRTC give you by means of high priority threads.

In retrospect it's a confusing term; we should probably avoid using it :)

disclaimer: use of the word realtime in the Matrix.org blurb is all my fault...

At least Apple's failure to deliver on their promises on Open FaceTime leaves an opening for other folks to deliver (e.g. http://jointheseen.com/sdk)