Taking e-mail back, part 4: The finale, with webmail and everything after
arstechnica.com
arstechnica.com
The simplest self-hosting solution I've found is to setup 2 cheap VPS servers that accept mail via postfix (128mb, maybe 256mb if you want to virus scan at this point, a good idea), and they both can deliver email to an in-house server running roundcube or zarafa or whatever, when the internal server is up and running. As long as the VPSes are up, they hold email until the server inside your local network is up and running.
This means you bypass the very messy "secondary mx" issues, as you only have one internal server to worry about that accepts email arbitrarily from the 2 public server, and you can backup your internal server just like a normal PC.
I had a similar setup for a while, and it was interesting to see that some messages (spam, mostly) picked the secondary MX by default. Spam servers also seemed to cache the (secondary) MX for much longer than the TTL.
Incidentally, if you are trying to read email or send email with code, having your own email server, working with a desktop client is a very useful starting point. You can downgrade the spam filtering, the SPF record checking, the SSL/TLS sign on until you have something that can read IMAP or POP. This is where the real fun begins as emails come in various multi-part formats, there are lots of character sets to translate to UTF-8 and there is base64 to deal with. If you can reliably read the header of an email and extract something simple like the date then you are doing very well.
It also does very good if not excellent job at illustrating how hard it is to set up, secure, and maintain your own email service. There are lot of places to go from here, including multiple MX'es, backup/recovery, etc.
Nevertheless. I've been using my own email infrastructure for almost ten years years. For last four I'm totally over it. Yes, there are some things to do if you absolutely want to be free to chose your email provider whenever you want to - having clear choice where to go, having backup off service provider, having migration strategy in place, etc.
No, we do have this capability: use XMPP.
In fact, XMPP is functionally enough to completely replace email, and does also serve as IM.
The reality is much more of a mess, with multi-device issues -- routing issues -- federation issues, not to mention a ton of non-XMPP messaging services.
Another issue is number of tools and skills we need to implement email server locally, which I suspect is a result of same specifications, protocols and formats.
I am no expert on this (not a real one anyway) but there has to be a point when effort to support a system outweighs convenience?
Other than that, I still don't see what your criticism actually is.
All of that is not to say that the email protocol stack is somehow particularly elegant (it most certainly is not), but most often when you hear people say that we need to reinvent email, they don't actually understand what the problems with email are and just seem to think it's bad because it's old--all while being completely oblivious of the lessons that have been learned with email. Despite its lack of elegance, it does actually incorporate a lot of wisdom, as well as its fair share of stupidity.
But what tends to be overlooked in particular is that for quite a few of the most obvious problems with the usability of email (namely, authentication of senders and spam), those problems are not problems that the people who invented and extended the protocols and formats failed to consider, but that despite a lot of effort and intelligence applied to them, they failed to come up with a solution for. And the reason for that tends to be that those problems are fundamental social problems that just crop up in email just as they do elsewhere, which can not be solved by simply inventing "modern email".
As for how difficult it is to set up your own mail server, I don't really have a clue. I mean, I do it all manually, but that's probably just because I learned how to do it quite a while ago, and now that I know how to do it all manually and I have a working setup anyhow, I don't know what easier ways might be available. But in principle, I don't see why it shouldn't be possible to put together a package of a pre-configured mail setup for standard cases which shouldn't be all that difficult to deploy. I mean, if you look at how many magnitudes more awful the web stack is, and ordinary people still manage to install and use a web browser, completely oblivious to the hilarious complexity of that piece of software.
I think the reason some people talk about starting over is because of this. The minimum level of security of email must be allowed to fall back to is "no security."
Other than that, I think those are all examples of problems that are not actually email problems.
Whether you can force TLS/STARTTLS doesn't really matter. First, you cannot ever force anyone to keep something secret, they can always publish a secret that they know somehow and thus make it not be a secret anymore. You can always offer STARTTLS though, so if the other side wants to keep a secret secret from eavesdroppers, they then can do so. That does not help against People in the Middle, of course, as they can strip out the STARTTLS offer. But being able to "force" STARTTLS doesn't help you either, as that would only force the middleperson to speak TLS to you. If you wanted to force authenticated communication that protects you from MitM, you would first have to set up some authentication mechanism. On a server-by-server basis, you can already do that. If you wanted to do it globally, we would need a reliable global PKI. Such a thing does not exist and is extremely hard to build--and in particular, it's not an "email problem". Reliable authentication is mostly a hard social problem.
As for metadata encryption: Well, that pretty much is impossible by definition, except maybe through mix networks. You cannot simply encrypt the information that needs to be readable in order to route the message to the destination. Even if you deliver a message via TLS to my MX, the mere fact that your mail server looks up my domain MX and then connects to it already tells an eavesdropper all the meta information there is. And if you solve that by instead doing all the routing inside gmail or some other big provider, you obviously haven't solved anything, as now google can see, sell, and be subpoenaed for this information.
Also, "security" is a word without any meaning. You can only be secure against particular attacks/risks, and securing a system against a particular risk often is in conflict with securing it against another risk, so you can only ever achieve one or the other, and it's often a difficult decision which risk to take. In the case of email, you could very easily secure the system against spam, for example: Just have a government agency that licences email server operators and shuts down any that engage in spamming. The side effect: Email is not the slightest bit secure against government censorship anymore. And yet again, there is a social problem at the core, not a technical one.
There are reasons why email is so "insecure". And as I wrote above: It's not that people didn't think about it. It's more likely that those who suggest that starting over could help solving the problems haven't really thought through it. In particular the "security" problems of email are extremely hard problems, and they are hard for social and political reasons, not for technical reasons.
Yeah, CA system is shit. but they are better than nothing. I don't trust it won't be compromised by a government NSL, but I trust it enough that I'll pay for ebay crap over SSL at a coffee shop and not worry that the guy next to me is sniffing my credit card number.
Here's what starting over means to me: you don't break something that basically works. If you start adding incompatible changes to email, then it splits the network. Instead create a parallel network with minimum standards attached to it (for instance,) and maybe nobody will use it but it also won't break anything.
We can make any communication channel secure, by adding layers of encryption, no problem with that. But unless it is defined deep down as a part of the standard, it all ends with no security at all and no means to be sure what happened with the message along the way.
Having headers exposed is one of the reasons why I raised initial concern. It is part of the specification and unlikely to go away by building on top of the standard.
And no, when you claim that HTTPS has "not many optional layers in between" that mostly suggests that you just treat it as a black box rather than that there is no complexity there. The simplicity is in the user interface only, and that is arguably broken because it does not actually reliably reflect actual security.
Really, once again, you mostly make the impression that you don't really understand the problems with email and just think that it's broken because it's old. Anyone with that attitude is likely to just repeat all the same mistakes that we've been through with email, and in the end not solve any of the actual problems.
I can't be any more clear than what I already said. If you decide to take an easy road and kill the messenger, no problem with that. I don't have any intention to continue with this discussion indefinitely, especially if it turns out to ad hominem. History teaches that killing the messenger does not make problem go away. You see it or not, decide to ignore it or not... fine by me.
And while you are at it, maybe also look up what "killing the messenger" means and how it is used.
But you are right that the much more fundamental problems that are actually there won't go away if you keep ignoring them.
By the way, the one who claimed that you were not really an expert on this was you, a few posts ago.
Now, if you have any actual specific problems with email that you think a new system would be better able to solve, feel free to explain it, then I can explain more concretely what the problem with your proposal is or how you are missing the actual underlying problem - or possibly accept that you indeed do have a valid point, though I wouldn't be too optimistic about that at the moment.
Email servers don't have to be complex. I wrote an SMTP and POP server in a single afternoon in the late 90s. The protocols have lot of optional parts, and if you want to get something quick up and running, there are simple implementations possible. The standard Unix servers are quite complex to administer but that's mostly because they were built for Unix system administrators. I mean, BIND configuration files are also complex, but that doesn't mean we should throw out DNS just because it's not trivially simple to implement.
The only point to reinvent and reimplement everything is if you can make some revolutionary improvement, otherwise, we should just improve what we have.