Streams: Mail 3.0 concept
tonsky.me
tonsky.me
It's hard to believe people don't understand that 'shortcomings' are sometimes 'advantages'. Email is perfect for what it is, it's a textual, non-realtime medium of communication. You take the time to compose a long, coherent message using written language. Mail has existed for a long time, and hopefully it will never go away.
It's great if you want to invent something more instant, more automated, more visual, but it's not going to be an improvement on email, it will be something else that fills some niche.
Email is the single biggest vector for "hacks" and stealing money. Email is the biggest vector for spawning botnets. Heck, in half the email clients I see, there's no easy way to verify if <a href="phishingsite.com">your bank login</a> is a valid link to your actual bank or not.
Plus privacy issues, auto loading images (tracking pixels), accidentally reply-all, the ad-hoc way people think adding 50+ people to a CC list is a "mailing list," the way people think reply-all "+1 Adding Charlie" to the 50 people to add one new person to ad hoc email chain from hell is acceptable...
It's not good. It's just everywhere and people have accepted productivity killing social constructs around it so they can attempt to communicate at about a 60% effectiveness rate through such a broken medium.
Also notes, contacts, file transfers, reminders and more.
Spam? Spam is everywhere and if it is not now, is because it's not popular. Once it gets to the level of "email", they will make sure you are getting spammed.
What bothers me is that people look for alternatives to make us more "locked" aka "safe" instead of trying to educate. We, devs/sysadmins, have to find ways to make sure that people know how to use and protect themselves not to hijack the tools they use/work/need just because we like to be EDGY.
And that's also what makes the alternatives utterly useless. It takes a huge advancement to unseat things like email. Even a new technology like Dropbox has powerful incumbent "defenses" against superior alternatives like Syncthing and Bittorrent Sync. Who am I supposed to use these new replacements with? Literally nobody I know uses them. I can't even use rsync to transfer data to and from my clients (not in the software industry).
> Email is the single biggest vector for "hacks" and stealing money.
If this were less vague and thus actually falsifiable, you could also say that the single biggest vector for hacks and stealing money is the internet. The reason that email is the main vector on the internet is its ubiquity.
It would be great if email had been originally designed with signing and encryption capabilities, but it wasn't. The reason it is still better than some new messaging service that includes these capabilities is that is federated and decentralized and very tolerant of network failures.
> communicate at about a 60% effectiveness rate
Honestly, if humans could communicate 60% effectively, our experience as a species would be completely different. I don't think that email is the bottleneck here...
Yes, the protocol seems dated, but it is well-understood and implemented by thousands of vendors. It's a miracle that we were able to agree on a protocol, and on top of that, agree on a protocol that is relatively decentralized (although it does depend on DNS if you prefer not to use an IP to email). It isn't like Twitter or FB where a single vendor controls the platform.
It wasn't that miraculous, actually. Remember that it was invented before HTTP even existed. It was developed at a time where the only contender was Usenet. Internet was not yet used for any commercial purposes; there was no concept of platform. Internet, and thus email, was the medium between universities that we know of. Sendmail was included by default in each UNIX from that moment on; when you connected to the internet, you could already send emails.
The conditions were very different, it doesn't make a lot of sense to compare it to what we have today.
Anything vying to replace it needs to capture these features. I won't trade reliability for real time. There are other real time channels I can use.
Seems like he's increasing the bandwidth usage for an MX in a big way.
That's djb's greatest strength and greatest weakness. He just does whatever he wants and doesn't care how it integrates with the rest of the world.
We shouldn't have a unique address but rather give a unique random token to anyone asking for our email address which could be severed (or bound to receive from a single address, ie that counterparty can not transmit that token to anyone as my server wouldn't receive mail from another address).
We should have systematic guarantees that the sender is the mailbox it pretends it is.
Encryption in smtp should be mandatory.
We should have an easy to use content encryption (pgp style).
Ideally you would like to decorrelate the address from the service provider (lots of people are prisoners of gmail, hotmail or their ISP).
I am sure we can make the chistmas list much longer but to me these are the major issues with the current system.
Address exhaustion is a real issue though, so you will need to acquire many email-worthy domains for this service.
As you mention we can use aliases for unique emails (which is what I do, but I had to create a software on the side of my mail server to manage them, as my mail server never intended to use aliases that way, and for instance makes the list of aliases visible to everyone on the server, and does not allow to assign a keyword to an alias).
It is possible to have guarantees on senders by using spf and dkim, but not many server actually use them and you currently can't make it a hard requirement for your incoming traffic, so pretty much useless.
Smtp encryption has taken a step back with TLS, as with STARTTLS we are now forced to use optional encryption that can be downgraded by a MITM, whereas before one could elect to only have an SSL port open.
Content encryption is possible using PGP but clunky and almost no one uses it.
And people can buy their own domains if they know it is possible and how to configure a DNS server, needless to say not many people do.
What I suggest is that we would need a protocol that does all that by default.
>> You operate on a level of the entire messages flow. Forums, chats and web 2.0 networks are trying to make conversations feel as lightweight and barrier-free as possible.
... caught my eye. I manage a team of younger developers for whom real time chat is simply the way they grew up interacting. Everyone is signed into hangouts and "operating on a level of the entire messages flow" (sic) all day long. A large part of the interaction is off topic, vague, and ineffective. Here's an example from yesterday...
Dev 1: the repo url is wrong, I'm getting a 404.
Dev 1: wait, might be a typo.
Dev 1: my bad.
Everyone on the team received this message and presumably stopped their current mental task at least long enough to read it, as I did. In the end it had no impact whatsoever. The developer clearly should have thought through his problem before notifying the entire team that he had one. But this is how chat is used.
I've tried very often to get them to be more deliberate in chat communications, without any success. I have also tried to encourage complicated topics to move quickly to voice hangouts, with even less success. Streams of real time chat messages remains the primary way the group interacts, to the point where I have backed off my efforts and come to the conclusion that I'm likely the one who has to change.
That said, there will always be a need for deliberate communications within organizations. Email grew up as an analog for letters and memos, which served as deliberate, archived, time-stamped records of an important communication. They were considered, usually minimal (because they took effort to create), and got to the point. They were events, not streams of ephemera. I'm not sure that this constant buzz of chats does a good job of replacing that. So for that reason the sentence "You operate on a level of the entire messages flow" doesn't make me smile.
1) giving E-Mails more inherent structure that is managed centrally (streams) instead of by each user (filters, folders).
2) making E-Mail more synchronous instead of asynchronous.
As far as I understand, you're arguing against (2). I agree with everything you wrote, however I still find (1) a very good idea. It is also an idea that IMO could be implemented entirely without changing the protocol - it's just a question of where you put this additional information. Streams could also simply be represented as a group address in the CC field that is managed differently by their client/server architecture, but otherwise fully downwards compatible.
It's not.
The reason those things haven't been dethroned is they are he lowest common denominator. You can't dethrone them by making something better. You can only dethrone them by making something that is both better and more common.
By definition the lowest common denominator is crap. By definition it is some legacy junk that was already on every computer 10 years ago. By definition it's a hundred competing standards from multiple companies because how else will a technology work on every device ever?
You can try to replace it with another tech, but it's going to have he same kind of characteristics, and it's arc of adoption is going to look like email's. Which means maybe 10 years from now it starts to look like a real contender. And by then how much crap has been jury-rigged into the spec?
It's the long tail. It's janky, but it's a common standard that does one job like a freaking champ. The long tail will never be engineered away. It ebbs and flows. Only in the last 10 years is snail mail losing its status as a reliable way to reach people.
What if it was a no-brainer to publish to and subscribe to feeds, there was some kind of feed exchange and access control built-in, and everyone was using them to communicate? What if Slack was open and decentralized, and everyone was using it?
We tried that; it was called IRC. As far as I can tell, Slack is still little more than a hosted IRC service with its own web and mobile clients. Virtually none of the functionality they provide couldn't be replicated this way; it's more a matter of branding than anything else that they eschew the label "IRC".
Slack even provides an IRC gateway so you can access it using IRC clients. I don't know if their web and mobile clients are using the IRC protocol, but from what I can tell, they might as well be, which is the whole point.
And as always, there is a huge gap between "X is just A+B+C" and "Here is an actual X". Slack may have not done anything on the transport side, but they did make the client better. AFAIK the only IRC client with full text search on the whole history, with support for files and images, is irccloud.com. That's not a lot. I wouldn't say IRC does everything Slack does.
IRC ain't dead yet. For example, there are IRC channels for many of the projects within Mozilla:
IRC does not have this. I've to connect to your server, or you to mine. This doesn't scale as an email/IM replacement; it only works for rooms with a certain usage/target.
The huge problem with any replacement (aside from federation) is the network effect. No matter how good your solution is, you need critical mass for it to pick up, and no huge company is going to invest into getting users into a federated network which they won't control. There's no profit for the company.
IRC most certainly does have this. All larger IRC networks are exactly that: Networks of federated servers. It's been like that since the early 90's, when it was an absolute necessity as none of the ircds could handle all that many individual users.
Indeed, it’s the main catch. If at certain point ago critical fraction of people (‘everyone’) all were using X for what we currently use email for, be X IRC or anything else, then X’d probably be the today’s email.
Services such as the aforementioned Slack seem to be going maybe the only pragmatic ‘email disruption’ route. In corporate world you can have the central authority say “everyone use this or you’re fired!” and thus bypass the network effect barrier.
mailto:(code-review|off-topic|blog)@tonsky.me?subject=[un]subscribe
Sounds more like he is suggesting improvements to mailing list interfaces than email itself.
This exists in rather decent form in Google Groups already -- there is Google Groups for Business (with a Google Apps account) that gives you a great web interface as well as email interface all just for your company.
http://www.renepedersen.com/wp-content/uploads/2010/01/googl...
If I had 10 pence for every "email is broken" post, I could have had a free pint down the pub.
Anything is hard to manage in huge amounts, and anything real time is harder to manage even in small amounts.
What I think With that in mind, I feel this concept is very close to a lot of the p2p or decentralized content distribution systems being tested out these days, like aether[1] or matrix.org[2].
The backend implementation that matrix.org uses is rather ideal. It's Federated JSON over HTTP(S). The clients out today are heavily focused on chat, but it is data agnostic. You could have client that writes rich media content and a client that displays this as a blog. Going back to the streams concept, you could have a client subscribed to multiple streams (rooms in matrix vernacular) and as people post their messages - rich media or simple "how are you" comments, it would get federated out to each home server, which can present a web client interface like in tonsky's mock-ups.
For the concept of amazon providing sensitive order information as a stream/matrix room, there would have to be an add-on concept built into the client of some sort of security token. I'm not talking about a losable gpg keypair, but something amazon generates and provides to you (like they do with their AWS services already) Amazon would then create a personalized stream for you, using that security token to encrypt. Since amazon provides the content and the key, they can replace your key at any time.
[1]: http://getaether.net/ [2]: http://www.matrix.org/
And now with Aether - has no one heard of freenet and i2p and Tor? No, we'll re-do all the sort of starting work on anonymity and security and get to re-make for the thousandth time the mistakes where it really isn't that secure or anonymous rather than building on anything that has at least some of the actual battle testing and bug squashing already done.
What is Aether but Frost with less testing? Hell, what is it but TOR access to Usenet? Maybe I don't understand what's so special that it can't use any of the long worked on infrastructure level work that's gone into the above projects for a decade or so, and it actually does make sense to start over from day one - but if so, they certainly don't explain it.
And I'm not talking about UI - I'm talking about the anonymity equivalent of deciding TCP isn't good enough so I'm going to re-invent IPX or something.
Most social services we use today could be streams. Facebook, Twitter, even WhatApp, Instagram (I guess not snapchat). If they'd all implement some common stream spec. Unfortunately they won't, Twitter has demonstrated that they don't like alternative client implementations.
If it could replace e-mail is another big question. Nikita hilights the amount of cruft that's around e-mails, but the cruft is social protocol, and we humans love social protocol. Or perhaps that's just old fashion speaking.
The posting is quite old already (Oct 2014), so this isn't some big hype building and it doesn't look like anyone is working on a big implementation.
Technically, the standard XMPP PubSub protocol with its PEP companion could get you very far already, unfortunately there's not many clients that handle them today.
Well, at least it doesn't live inside a silo, like most other messaging protocols.
Also, issue tracking is something that could be replaced with a stream...
These are "limitations" according to the author. But IMO they're the reason I like email:
- I like to be able to reply when I want, without the sender having the expectation of everything happening in real time, I have other things to do than answering real time to everything.
- Anything is hard to manage in huge amounts, which is why we hide less important stuff. Email has folders where you can put stuff in. You can archive emails. Or maybe this management issue is technical, as in "storing petabytes of emails is a difficult task" ? In that case, I have no real answer.
- I actually like the concept of mailing lists: they work where I already am (ie my mailbox) and I can stay up to date just like in a forum. And person-to-person dialogs do indeed work very well with email. I don't see that as a limitation.
---
Also, about this: "Stream is a group of threads. Simple as that."
That does not sound simple. In fact, just from that sentence, I didn't understand at all what it meant. I had to read on to get the idea. But I do see the point of "streams", and I agree that filters are usually complex to setup and maintain.
---
The part that I don't understand is this one: "With proper user/access management Streams could replace not just email, chats and group chats but also forums, blogs, mail lists, RSS and most of web 2.0 communities."
It sounds like what you need is RSS everywhere. RSS has a pull model too. You can setup ACLs on an RSS feed if you have some kind of authentication in place. It can replace mail lists, except that you'd still need to be able to push messages to it, which I don't see anything about in the blog post.
---
Finally, regarding this: "This fixes ambiguity around do-not-reply@ addresses"
I get the point of ambiguity and I have replied to no-reply addresses before myself, with an error message coming back. But I think ambiguity could be fixed with a header here, instead of throwing away the whole email infrastructure.
Imagine a "X-No-Reply: true" header. That way, email clients could disallow replying to such an email address. This has the advantage of being backwards compatible with everything that exists.
---
Overall, I do think that you can try something new, trying to "replace" email. But realistically, Google has tried several times and is struggling. Email has solid tooling and knowledge built around it, and I think trying to push for its self-improvement is worth a look.
Slack is sometimes said to be an "email-killer", but the fact is it replaces only one use of email. At my last company, we used Slack for real time stuff, notifications, quick questions, but not for everything. And that's great. Because email sucks for real time stuff right now (although initiatives like JMAP may make things better).
Email seems to carry the notion of importance, which is not true of chat messages I think. The reasoning behind this is, I think, an incoming email stands alone, whereas a chat message is quickly surrounded by other messages.
Also, emails have a clear receiver, the receiving email address. This means that if you get a message to "devs@project.org" and you are a dev, you are de-facto concerned by this email, which is a reason to read it thouroughly. This is not true of chat messages which do not have a "receiver" but are instead posted to the more liberal notion of "channel".
This is critical. Understandably, "chat" user interfaces don't make individual messages stand out by themselves. This means that it is easy to miss a message, or even a conversation. It also means that you can't delete everything but the important stuff you need to reply to, which is trivial in email clients.
I don't mean to steal this thread, but if anyone is interested in building the future of email (including this) I'd love to talk with you. mg@nylas.com
Edit: To see a fully functional version check out http://plugg.io
Also, let people publish Streams as Streams and non-Streams users can subscribe to those Streams as email.
And right there, is why it won't catch on.