STARTTLS Everywhere
starttls-everywhere.org
starttls-everywhere.org
[edit] actually looking at the RFC it relies on a TXT record before the https policy is even fetched.
BTW, what ever happened to SMTP on a dedicated TLS port (465)? Why did it get deprecated?
So it makes sense to deprecate this use of port 465 and just tell people to use STARTTLS.
It's OK, lots of people don't know about this. Really the only problem is that you've assumed since you know nothing about this you're an expert, that'd be Dunning-Kruger or a related effect at work.
For submission it really can work to upgrade end user mail agent software. We're a long way from useful encryption capable agents being ubiquitous, but we've made progress and most popular systems today ship with such a client. These systems also use TLS directly for IMAP (to receive email) and can share a lot of logic across the two systems, so that's nice. In maybe a decade we'll be where HTTPS is today. For submission.
What distinguishes a submission port is that it's authenticated. You can only send email via the submission server as Alice if you prove to the submission server that you're Alice.
But SMTPS (obsolete SMTP over TLS on port 465) and the STARTTLS SMTP (on port 25) are for email relay. Authentication makes no sense in this scenario because the relays are neither sender nor recipient, the transmitting relay can't prove that it's Alice, and the receiving relay can't prove that it is Bob, because they're not, they are merely relays.
These are servers, so they upgrade much more slowly (often only on hardware replacement) and the priority is reliability of delivery, not security, if your proposed solution drops 1% of email on the floor that's far too much and nobody would adopt it.
Hence the situation we have today, and hence STARTTLS Everywhere. Your imagined alternate universe makes no sense except because you've misunderstood how email works.
1. Senders should first try TLS-on-connect, and if it's not available, fall back to STARTTLS.
2. Senders should only do TLS-on-connect with no fallback
If you do #1, then a downgrade attack still exists because an attacker can just block port 465 and then interfere with the traffic on port 25 (or put a fake SMTP server on that port if one doesn't exist).
If you do #2, that's no easier than just mandating that STARTTLS on port 25 is required.
The alternative to all of this is to find some out of band method for a receiver to state that TLS should be available (TLSA / MTA-STS), and if it's not then something is being tampered with. That way, a sender can look up that information and then choose to do #2 for that particular receiver. Again, in that scenario it makes no difference if we're talking TLS-on-connect or STARTTLS. Given we can't make the whole World switch to "must always be encrypted" at the same time, this is the only practical way forward.
As others have explained an on-path adversary who'd be able to rewrite packets in order to stop STARTTLS can even more trivially block port 465 altogether. In fact cheap appliances can do the latter but can't do the former, so if anything STARTTLS very slightly improves this.
If you are imagining "Well I could have a policy of only using port 465" well, sure, but you could just as easily have a policy of requiring STARTTLS. In both cases an adversary can deny you, and your only options are to re-think your policy or give up and not deliver email.
I guess that's the problem - there is no practical way to disable plaintext email from the start. You may always need to accept the connection, see if the client will STARTTLS, disconnect if they do not and hope they don't re-try and keep hammering your servers with the same message, the error message "hey, i'm not accepting plaintext" will most likely get ignored.
The key advantage of MTA-STS is that it can be deployed quickly, without almost any disruption of the already-deployed infrastructure. Of course DNSSEC and DANE already solve the same problem, but MTA-STS is the designed by and for people who don't want to use DNSSEC.
The EFF's efforts are complementary to MTA-STS. As schoen mentioned elsewhere on this thread, in the early days it's probably useful to have a preload list. (That said, I am not sure why they feel it necessary to cast MTA-STS in negative light: "[...] so the sender will never know the recipient supports STARTTLS." [2] They should explain that there MTA-STS is trust on first use and has a memory effect, exactly like the well-accepted HTTP Strict Transport Security.)
We added support for MTA-STS testing to Hardenize just today, along with a blog post that explains the background and explains how to deploy it. [3] (Disclaimer: Hardenize founder.)
[1] https://www.ietf.org/mail-archive/web/ietf-announce/current/...
[2] https://www.eff.org/deeplinks/2018/06/technical-deep-dive-st...
* The SMTP RFC says that mail servers MUST NOT require STARTTLS to receive mail. Postfix (and I imagine most other production grade SMTP servers) has an option to require STARTTLS anyway, so if you really want STARTTLS you can already require that clients have it enabled, despite the braindead standard.
* STARTTLS ensures that mail was encrypted only in the final hop, from the last server to your server. That usually means it transited the public internet encrypted, but it definitely does not assure it.
There are interesting email security efforts afoot, notably the draft standard called "SMTP Require TLS": https://tools.ietf.org/html/draft-ietf-uta-smtp-require-tls-... . Unfortunately the reality is that the internet mail infrastructure evolves at an incredibly glacial pace. The entire SMTP protocol would benefit greatly from the adoption of an SMTP/2 protocol, rethought with modern security practices in mind.
With many ISPs (and even some VPS) now blocking port 25, it could have been an opportunity to migrate naturally towards the "new" unencumbered port.
The email RFCs require supporting a lot of things that are now horrendously outdated, and forbid doing a lot of things that should now be best practices.
The RFCs should lose that battle.
* http://jdebp.eu./Proposals/IM2000/
But of course you are already using a pull-style messaging system, that is right in front of you.
Moreover it has the advantage no other popular communication service has: people somehow managed to agree on protocol standard so anyone is able to run own smtp server. In modern days of huge corporations a new popular standard like this seems impossible.
We are managing to replace http1.1 with http2, it will take time but it is on the way. I am not even aware that an actual SMTP2 draft protocol that would solve all these design flaws (unverified sender, unencrypted).
And SMTP has a benefit http didn’t have. Most people access their emails through webmails, smartphones or enterprise outlook/exchange servers. For webmails and exchange only the server needs to be updated, and for smartphones, their short life ensures that older versions are pretty much all retired within 5 years. In addition, a handful of big players (google yahoo apple msft) have such a concentration of recipient accounts (retail users) that they can force a change on the market with the threat of your mail being classified as junk by them. So we could achieve a pretty quick transition.
* https://news.ycombinator.com/item?id=17395600
SPF, of course, should be used everywhere so that we reach the point of complete balkanization and collapse, and people realize that we can (and in fact in some ways already have) switch wholesale to different systems, faster. (-:
- pre-delivery forwarding servers just need to be added to spf. If you use random third party smtp relays, then this is precisely what spf is trying to avoid.
- the way internal servers implement aliases is their problem, there is not necessarily a need to go through an smtp relay (my mail server doesn't)
- failover mail servers should check spf on incoming email and then have a trusted relationship with the primary server so that spf isn't enforced when the failover delivers to the primary (that's the way my mail server works)
- spf uses DNS. So what?
- ISP lock-in. If you control the domain/DNS entries, there is no lock in. If you don't, then you are already locked in anyway.
- doesn't allow dynamic IPs. I'd argue that 1) it is a good thing 2) it's not really the case, you can specify a domain in spf, and this domain can be a dyndns style domain with a short lived TTL resolving to your current dynamic ip. And in theory you could also dynamically update your spf as your ip changes with a short TTL (like a dyndns-style entry).
[edit] and actually what is going to kill you with a dynamic IP is not so much spf than the fact that the reverse dns of that IP won't resolve to your domain which is a big spam red flag for most smtp servers.
The huge problem here is "forward all e-mail I receive to my gmail account" or similar. If you do that pre-delivery, you break SPF. If you do it post-delivery, gmail makes you responsible for any junk that gets through. If you make your filters harsher, the user complaints some e-mails are lost. What is your solution to this problem?
> the way internal servers implement aliases is their problem, there is not necessarily a need to go through an smtp relay (my mail server doesn't)
Now you send an e-mail to contact@yourbank.tld and the message is rejected because of your SPF policy and their usage of internal relays. You can (a) fight the corporate shitshow to get someone to fix the bank's relay; or (b) relax your SPF policy. You may be willing to pursue (a), but a company that sells e-mail services to thousands of clients just cannot enter those fights and still be economically viable.
Don't get me wrong. I use SPF (DMARC actually) on my personal server and it actually helps (as a low volume sender), but the moderate volume senders' problems are different from those of personal e-mail servers, and SPF works much worse there.
I'd argue this is the fair and correct behavior. Effectively you created a kind of open relay server. Since you accept external traffic, you should be filtering for spam there.
> the message is rejected because of your SPF policy and their usage of internal relays
I'd assume that if it is an internal relay server, then it shouldn't be checking for spf internally. Only the server receiving incoming external traffic should check spf. Sounds like a misconfiguration.
Maybe it would help to allow users to whitelist such cases.
A behavior that breaks how the mail system has been working since forever, and that people expect and use all the time.
It may not be right, but it is used and not even the big players (such as gmail) are willing to break it (hence why gmail actually tells other admins to use pre-delivery forwards disregarding SPF, but respecting DKIM wich is broken if you use post-delivery aliases) [1].
[1] https://support.google.com/mail/answer/175365?hl=en
> Sounds like a misconfiguration.
A misconfiguration you cannot fix (because it is on the receiver end, not on your end). But the client pays you for the service, and understandably asks you for a solution. What would you tell them?
I understand your points, and mostly agree with them. However, this approach only works in an ideal world where users understand that e-mail should have some limitations it hasn't had for the last 30 years, and all administrators are "good citizens" (they know their stuff, acknowledge their issues and work to quickly fix them).
The real world is different: clients will demand solutions, and other admins will oftentimes be either ignorant, powerless or even adversarial.
(Edit: in addition to the site tester that you noticed, there is also a public policy list and some forthcoming tools to enforce STARTTLS on the client MTA side when delivering e-mail, preventing downgrade and MITM attacks.)
However, you can also use Let's Encrypt to make this more useful by getting a publicly-trusted certificate for the TLS service on your mail server, and then listing your mail server with this list!
The introductions my colleagues posted about this today are at
https://www.eff.org/deeplinks/2018/06/announcing-starttls-ev...
https://www.eff.org/deeplinks/2018/06/technical-deep-dive-st...
which will hopefully give a clearer explanation of what the service is meant for.
SSL is not mandatory on either port 25 or 587, and SSL can not be made mandatory if you follow the RFCs (it can be made mandatory if you tweak your MTA, but you may lose some mail). Advertising for SSL over DNS means you trust the DNS records - which you shouldn't without DNSSEC. Even with it, there can be workarounds that in practice will allow MITMs.
The only real solution is making SSL mandatory, and doing SMTP over SSL as in the good old days of using stunnel on port 465 to decrypt, then netcat to forward the output to the MTA running on localhost:25
But that is not standard. Maybe the efforts would be better invested by changing the standards to have a port where SMTP can not happen at all without SSL - just like port 465 was, over 10 years ago.
I'm well-aware of the RFC difficulty, but I don't think that the current approach of STARTTLS Everywhere is really a problem because it's effectively opt-in on both ends. The enforcement is requested by the receiving side and then implemented by the sending side.
* The receiving MTA has to proactively choose to be listed.
* The sending MTA has to proactively choose to make use of the list.
So, with the current version of STARTTLS Everywhere, only sites that deliberately choose to enforce STARTTLS will do so, and they will only do it when communicating with sites that have specifically asked them to enforce it! This would only be an RFC violation if we thought that the RFC meant to categorically forbid sites from separately agreeing to a stricter security policy.
This approach might have its scalability limitations, but I won't try to speak for my colleagues about any future steps.
The problem is not the MTA who will chose to be listed, but those who won't be listed - the immense majority. "Scalability limitation" is certainly a more polite way to say that.
I'm sorry if my message was too blunt, but I am not sure it was worth downvoting my technical explanation just for this.
The HSTS preload list also has a huge scalability problem, but it's also improved the situation about HTTPS downgrades a lot!
DANE is being recommended (mandated?) by the European Union, and is on the rise. MTA-STS is backed by Gmail, Microsoft and many others, which is likely to give it traction.
To sum up from that post, we think STARTTLS Everywhere is a stop-gap measure until DNSSEC is fully deployed, and STARTTLS Everywhere can act as a preload list for MTA-STS (to prevent DNS downgrade attacks).
- Increase STARTTLS adoption
- Increase the number of mailservers that actually validate certificates
- Maintain a STARTTLS Policy List to help prevent downgrade attacks on email services.If we were talking to a bad guy then we're screwed. But if we are talking to our intended recipient, even if their cert is bogus, our connection is protected from bad guys on the path. The bad guys can't choose later to access that communication after all, their only opportunity is to guess we're doing opportunistic encryption and intercept the original connection to play MitM, and if they guessed wrong their hand is shown by doing that.
This situation is unpalatable even for nation state adversaries like the NSA, because they would prefer not to be seen to be meddling. Even for something like the FSB, which scarcely cares if it's seen to be meddling, it does force them to make a decision they'd rather leave until after the fact. MitM everybody (and be known to do it) or leave them alone and maybe regret that later.
You acknowledge that you don't know who you are talking to. That alone proves my point. If the cert is bogus/unvalidated then its trivial for the bad guy to intercept and supply their own bogus cert instead. You are correct that its encrypted, but that means nothing when you do not know who you are talking to. It could even be a string of bad guys all capturing and injecting another bad cert. Encryption without validation means nothing.
> they would prefer not to be seen to be meddling
Time and time again ISPs have been caught injecting javascript/cookies/html inside unencrypted traffic. If ISPs are willing to do that, then why should nation states be afraid to? The point of 'bad guys' is that they can't be trusted to not do bad things.
Besides the point that bogus/unvalidated certs do nothing, it would be easier to just perform a downgrade attack if the client isn't doing any validations. Downgrade to unencrypted and easily see everything. That is why they are creating a list of services that should only allow valid encrypted connections.
Then proceeds to show results for the MX anyway. With a single error: "Does not use a secure TLS version Error: Could not establish connection with hostname %!s(MISSING)"
Seems a bit broken to me.
Other than that this is really cool!
It made me sad though -- I checked all the domains that I used to run mail for at some point in my career, and not one of them passed all the tests. When I managed email, I always made sure that our servers met all the current standards.
The only problem being that that list is currently practically empty.
So while I was considering adding that email server which I maintain, when I saw the items on the list it makes me reconsider.
The only email servers currently on the list are : google, yandex, yahoo, icloud, outlook, comcast, eff, qq and facebook.
So while it might be an opportunity to get your server on the list early, personally I worry more about what it will break as to what it will help.
In conclusion, it looks like a great initiative, but it will have to be in production for a while before I would recommend anybody to add their server to that list.
See https://github.com/EFForg/starttls-everywhere/blob/master/RU... for more info.
Another question if I may.
The site mentions:
> the STARTTLS Policy List gives mail servers another point of reference to discover whether other mail servers support STARTTLS
Is there any email provider, email server component or email client that already uses this policy list?
Because that's another thing I miss from the FAQ.
We need to revisit the STARTTLS vs implicit TLS debate in light of the obvious vulnerability and overhead of starting with plain TCP connections and then hopefully signalling towards security. HTTPS is obviously implicit TLS and it works great. We know STARTTLS has issues. Can we not keep going down the STARTTLS road for email, while going down the implicit TLS road for other things?
The problem is partly because we don't have an assigned port for MTA2MTA implicit TLS. Otoh DANE also already provides a way to have mandatory TLS for MTA2MTA traffic.
If there aren't major technical obstacles we might be willing to take pull requests for STARTTLS Everywhere that allow mailservers to announce self-signing policies, but it hasn't been a priority thus far because LE certs are so easy to get and are slightly more authenticated.
Mail is very sensitive communication. It is reasonable for the EFF to worry the risk of evedropping. Some websites are still sending passwords by email!
But you can chose to disregard the RFCs and disallow servers that will not encrypt to send you any mail. No downgrade attack then. It requires some manual changes. It may cause you to not receive email from some servers.
You can also only accept mail on port 465, which in practice is used for SMTP over SSL. You will receive even less mails.
Cf another reply I made today about that: https://news.ycombinator.com/item?id=17397500
However in January this year, the proposed RFC 8314 reinstate the registration of port 465 for implicitly encrypted mail submission. I think it is a bit early to close down port 25, but a great idea to make sure 465 is correctly configured if it isn't already.
My server was dropping the connection after EHLO because they were connecting from a different IP address and specified -all in the SPF record. Maybe you are seeing a different issue.
He's entering "fastmail.com" into the form on the web site then the EFF's server is trying to connect to (at least one of) FastMail's MX hosts but is unable to establish a connection.
As OP said, what he is seeing has nothing to do with SPF.
(Side note: I'm a mail admin and I'm pretty strict, but even I don't drop anyone after HELO/EHLO. I'm not surprised you're having issues.)
Where the subject name is the name of a something on the Internet it should use the Internet's defined "Subject Alternative Name" (SAN) mechanism, rather than trying to squeeze into the X.500 system's hierarchical directory. For compatibility, and to save the X.509 subject being empty which confuses some software, you may write one of the DNS names or IP addresses into the X.509 Subject's Common Name (CN) field as human readable text, but you should always write all DNS names and IP addresses into SANs.
The dnsName SAN is defined like a DNS record, so it's case-insensitive and (if it's an IDN) Punycoded, it's also deliberately defined with a single encoding that is too narrow for anything much beyond an actual (Punycoded if appropriate) DNS name, to avoid people trying to write "extended ASCII" characters into this field by mistake.
You can edit the cron job for certbot and add a script that should be called when certificates were renewed.
See: https://community.letsencrypt.org/t/please-avoid-3-0-1-and-3...
If you set things up to involve CSRs this problem goes away because the CSR binds to a key pair, just don't change the CSR unless your DNS names change. But if you use CSRs, Certbot forces you to take charge of ensuring the renewal schedule etcetera, since it has no way to be sure if the same CSR can be used next time.
Mail servers can do more here to help automate this. Also, maybe we can imagine that TLS-ALPN-01 (a forthcoming Let's Encrypt proof of control method) could work with SMTP STARTTLS. That would let a mail server take responsibility for getting its own certs (port 25 is on the short list of "Authorized ports" for the Ten Blessed Methods) by telling its own TLS implementation "Hey, when asked for ALPN, offer this extra ALPN feature and repeat everything I tell you" to prove it is really who it says it is.
I think the above could work, but it would need effort from Let's Encrypt and TLS-ALPN-01 itself isn't finished yet.