Nginx doesn't suck at SSL after all
matt.io
matt.io
In the normal protocol, if you lose the RSA key, an attacker can retroactively decrypt the session keys, which are protected only by that same RSA key.
In ephemeral DH mode, instead of encrypting a session key with RSA, both sides run the Diffie Hellman protocol to exchange a key†. DH allows two unrelated parties who share no secrets to exchange a secret in public; it's kind of magical. But it's also trivial to man-in-the-middle. To get around that problem, ephemeral Diffie Hellman mode in SSL/TLS signs the DH exchange with the RSA key.
The win here is that losing the RSA key now only allows you to MITM future SSL/TLS connections. This is still a disaster, but it does not allow you to retroactively unwind previous DH exchanges and decrypt earlier captured sessions.
† DH is unbelievably simple; go read the Wikipedia page.
It sounds magical too.
http://en.wikipedia.org/wiki/Diffie%E2%80%93Hellman_key_exch...
http://en.wikipedia.org/wiki/Boyer%E2%80%93Moore_string_sear...
openssl s_client -host HOSTNAME -port 443
I ran this for my own website and a few bigger websites
openssl s_client -host www.gusta.com -port 443 (My site, hosted on Heroku)
Cipher : DHE-RSA-AES256-SHA
openssl s_client -host www.google.com -port 443
Cipher : RC4-SHA
openssl s_client -host www.airbnb.com -port 443
Cipher : AES256-SHA
openssl s_client -host www.facebook.com -port 443
Cipher : RC4-MD5
openssl s_client -host www.paypal.com -port 443
Cipher : AES256-SHA
openssl s_client -host www.amazon.com -port 443
Cipher : RC4-MD5openssl s_client -host online.citibank.com -port 443
Cipher : RC4-MD5
openssl s_client -host www.bankofamerica.com -port 443
Cipher : RC4-MD5
Nobody likes it, but until relatively recently, the AES ciphersuites were all CBC mode, which means they burned a couple bytes of padding for every record.
RC4 is also faster than AES, which was, until very recently, an issue for server performance.
We have AES-CTR ciphersuites now, but I'm not sure how widely deployed they are.
RC4 is faster than AES on older processors. On the current generation of Intel server processors, AES is faster because of AES-NI.
AES-GCM cipher suites should be even faster yet on those processors, but they are not widely implemented. Also, I wouldn't be surprised to see more interesting papers being published about weaknesses in the GMAC function in the near future. (Note: I don't have any knowledge of upcoming papers; I am guessing there will be some based on the recent paper regarding the discovery of unexpected weak keys in GMAC.) Nonetheless, I suspect that NSS will implement them because they are part of the NSA Suite B profile for TLS and some NSS developers want a complete implementation of that.
1) Habit
2) As implemented/deployed in SSL, it still provides some security
RC4 has gotten a bad reputation in large part because of its poor application in WEP that resulted in keys being rapidly recovered by sniffing traffic. The Wikipedia entry is a good place to start http://en.wikipedia.org/wiki/RC4#Security (& numerous references for the original papers/pubs cracking various bits of RC4). The RSA response to RC4 concerns (from WEP) is worth reading, as well http://www.rsa.com/rsalabs/node.asp?id=2009 .
Here's my vintage code for scanning SSL configs: https://github.com/b/tlscollect
Here are a couple of must read posts from someone who really knows his SSL business:
http://www.imperialviolet.org/2010/06/25/overclocking-ssl.ht...
http://www.imperialviolet.org/2011/02/06/stillinexpensive.ht...
It's great to learn.
Lil' B
The "tradeoff" in security vs. performance you're referring to irrelevant to almost everyone building on nginx. If you've lost your RSA key, you are well and truly fucked. DHE is interesting, but sniping at people for not using it (in your case, implicitly) is unfair.
Adam - "However, with a pure RSA ciphersuite, an attacker can record traffic, crack (or steal) your private key at will and decrypt the traffic retrospectively, so consider your needs."
Matt - "Unfortunately, it also includes a very computationally intensive cipher using an ephemeral Diffie-Hellman exchange for PFS. Sounds scary already, doesn't it? ... The problem cipher is DHE-RSA-AES256-SHA [b]."
The first is factual and straightforward. The second is muddled and clearly skewed towards blindly disabling DHE. I believe we are in agreement that it is irrelevant to almost everyone building on nginx: their connection rates are so low they will not notice the overhead introduced by DHE.
I am sniping at enthusiastic ignorance and encouraging others to behave similarly. I hope that is all quite clear now.
Hugs and kisses, Lil' B
And we apparently disagree completely about DHE, because you appear to be saying you'd recommend it to web startups, despite the fact that the bank that clears those startups transactions isn't even using it.
Especially weird given that Boundary, your startup, doesn't do DHE.
Recommending people unfamiliar with configuring SSL leave defaults alone is only incompatible with our having non-default config if you are implying I don't understand configuring SSL. I doubt that is what you mean, as I am ever the optimist.
Yay!,
Lil' B
I'm having trouble parsing the rest of your comment. I don't have a religious belief about what defaults are reasonable to muck with and which aren't, but: this particular one is fine to change.
Matt simply says "I messed with my settings and leaving this one out makes it faster", without knowing whether or not turning DHE off is safe (or if he does know, clearly he's making it seem like he doesn't). The fact that it is safe -- in this instance -- isn't particularly relevant. The point is that someone who doesn't understand the security implications of something is making a recommendation about security, just cloaked in a recommendation about performance.
Anyway, I don't know any of the people we're talking about here, just trying to help clear up what I believe benblack was trying to say :)
"The win here is that losing the RSA key now only allows you to MITM future SSL/TLS connections. This is still a disaster, but it does not allow you to retroactively unwind previous DH exchanges and decrypt earlier captured sessions."
"If you've lost your RSA key, you are well and truly fucked."
Thanks for clearing that up!
Make love not war,
Lil' B
I agree that both articles have different degrees of technical completeness, but really getting snarky because you believe that someone doesn't have the 'stripes' to write an article that ultimately agrees with the former article you're comparing it to seems to me like a waste of time. Specially since both articles basically come to the same conclusion. It gets worse when you pull two quotes from tptacek that ultimately don't change neither what he said in the beginning nor the reason he's responding to your posts.
Either way -- based on his attitude in the first post, I'm really surprised by how Matt owned up and did his homework for this one. (He should have done it from the beginning, of course, but none of the people bashing him in the previous thread actually provided anything to support what they were saying.)
Usually when first engaged, you deal with operational issues (making sure all the applications they know about are assessed), but as you build on that, you try and instill secure development practices (so that every new application they build doesn't have the same issues as the ones you've just spent months uncovering).
The number of large clients I work with who don't have any SDLC process is staggering (I'd say it's the overwhelming majority of them). For the most part, the small group of security people are tasked with trying to secure the multitudes of applications which in many cases are 20-30 year old codebases. Their developer groups may be completely separate (usually from the result of all the mergers of financial institutions) and it's basically all fiefdoms.
As you start to work up the pyramid of enterprise security "hierarcy of needs" you get to things like a secure development life-cycle, but not all organizations are "ready" yet for that type of work. Some are just trying to figuratively stop the percieved bleeding.
A bank will have architecture and security teams that evaluate the applications, but their main job is to run each application through a "best practices" checklist or audit to identify potential trouble spots. An application will need to meet some kind of sane minimum requirement for password security, but many of these apps are legacy or mainframe, and not easy to change. Big banks move very slowly.
This tendency is independent of the fact that these functionalities are implemented by different teams, and that one team might happen to be competent enough to do the right thing despite the lack of institutional imperative. So the consumer might get lucky. So what?
Your initial comment was that "plenty of financial institutions" do SSL a certain way, and the respondent correctly pointed out that this fact adds no information to the discussion about SSL techniques, because plenty of financial institutions do dumb things. It's "apples and oranges" only insofar as he's saying the orchard manager is a poison spreading dummy so you can't trust the apples or the oranges.
Maybe you can elaborate on the "studied" part of your comment with specifics. That part was interesting.
Unless I'm reading OP wrong, that's not the case for the servers he uses for his tests: Stud can't enable it at all:
> stud doesn't have at all.
and in stunnel you have to compile it in for support, it's not just "not enabled by default", it's not compiled in:
> stunnel has it as a compile time/certificate configurable option.
Put:
if(getenv("SSL_CIPHER_SUITES"))
SSL_CTX_set_cipher_list(ctx, getenv("SSL_CIPHER_SUITES"));
anywhere after SSL_CTX_new().But don't bother doing it with stud, because (as I sort of predicted) stud already does this: stud -c <ciphersuites>.
I don't understand what Matt is saying by "stud doesn't enable DH at all". Does stud build its own OpenSSL? The system OpenSSL will already support DHE.
Here's a quick (no error checking) way to set up DHparams if they are appended to a cert:
BIO *bio = BIO_new_file(path_to_a_file_with_dhparams, "r");
DH *dh = PEM_read_bio_DHparams(bio, NULL, NULL, NULL);
BIO_free(bio);
SSL_CTX_set_tmp_dh(ctx, dh);
DH_free(dh);
Where do the DH parameters come from? You can generate it yourself (1024 bits here): openssl dhparam -rand - 1024
For a completely isolated implementation (requiring no user certificate changes), see function ngx_ssl_dhparam in nginx-1.0.4/src/event/ngx_event_openssl.cI had been checking things out in nginx and noticed that DH was not implemented. One quick email to Igor and he got it done the same day along with getting this into the next version of nginx. Dude is bad ass.
Now if only someone can convince him or provide a patch to add SPDY support to nginx...
If you try to use only DHE-RSA-AES256-SHA without DH being setup, nothing will connect. If you have DHE-RSA-AES256-SHA as an option with others, it will negotiate a non-DH cipher. (e.g. "DHE-RSA-AES256-SHA:!ADH:SHA" -- you can verify the ordering with `openssl ciphers -v DHE-RSA-AES256-SHA:!ADH:SHA`)
stunnel DH code inserted into stud.
Encryption Methods:
--tls (TLSv1, default)
--ssl (SSLv3)
-c CIPHER_SUITE (set allowed ciphers)
Edit: Though, you'd need to set DHE params, as another commenter said below. Stud doesn't do this atm, but I'm open to a patch!With AESNI, use AES-128, AES-256, RC4-SHA, CAMELLIA-128. Without AESNI, use RC4-SHA, AES-128, AES-256, CAMELLIA-128.
In nginx, this looks like:
# (wo/AESNI): ssl_ciphers RC4:AES128-SHA:AES:CAMELLIA128-SHA:!MD5:!ADH:!DH:!ECDH:!PSK:!SSLv2
# (w/AESNI): ssl_ciphers AES128-SHA:AES:RC4:CAMELLIA128-SHA:!MD5:!ADH:!DH:!ECDH:!PSK:!SSLv2
You eliminate weak ciphers. You retain RC4 for compatibility and speed. You order by performance. (Note that AES-128 is still ranked as secure through 2030 [at least]. You don't need to prefer AES-256.)Clearly the moral of the story is: "Don't claim that X sucks unless you are are damn sure".
Saying something sucks is fightin' words. Don't expect to people be nice if you are wrong.
> "My initial testing, which needs to be investigated
> further, is showing Nginx SSL performance lower than
> other alternatives."
You keep using that word ('headline'), but I do not think it means what you think it means.The result was his digging into it to figure out the default config included a computationally expensive setup.
That's a debatable thing. Not necessarily wrong.
I found the posts to be informative, but I get the impression that the "Nginx sucks at SSL" linkbait headline is what's rubbing folks the wrong way.
> Clearly the moral of the story is: "Don't claim that
> X sucks unless you are are damn sure".
More like: "If you say something that upsets people, they will spend a lot of time and effort to display their ignorance in an effort to prove you wrong -- as if you had insulted their very being -- even though you just made and honest mistake." I read the original article, and I didn't see it as an attack on Nginx, or Nginx's SSL support. He ran some benchmarks and said, "Wow, those numbers suck." > Don't expect to people be nice if you are wrong.
I read the HN comments here and on the original post, and I don't remember anyone saying anything about cyphers. The people that were 'not nice' were also wrong. Would that excuse the author of the post, from turning around and being 'not nice' to those people because they are wrong?Ouch.
Final feeling: My long time Twitter Followers & Friends are more engagement and kind than the anonymous programmers on HN.
It's also good to have thick skin. HN can be aggressive. But for good reason. I'd be willing to bet this tiny sting will result in more rigor in the future. I know it has worked that way for me.
before: nginx (ssl) -> haproxy: 90 requests per second
after: nginx (AES256-SHA with keepalive 5 5;) -> haproxy: 4300 requests per second
There is a Dave Chapelle joke about cops sprinkling crack-cocaine over a crime scene in order to make the case quick and easy. Too many developers trest SSL like magic pixie dust for security.
Or as ptacek says "thanks in advance for putting my kids through college."
Secondly, by removing the DH method do I restrict any browsers from connecting my site? Ie, are their any browsers, or security settings on browsers that prevent the site from being trusted if DH isn't available?
If you do not allow DH ciphers, you'll probably just lose users who deliberately configure their software to use only strong ciphers.
What you can do is put DH ciphers at the end of the list. That way, weaker ciphers are preferred but you're still supporting strong ciphers.
In regards to this post, if this is the default configuration of Nginx then I agree that Nginx sucks. This is not a good default configuration for the Internet.
Anyway, whether your phone has trouble coming up with the entropy, or preforming the math, you should probably be using something more substantial.
Particularly since at the current rate of development, the average phone will be able to do it just fine in a few months.
Burgerbrain: PayPal.com, CitiCards.com, 2checkout.com, and many payment processors all use RC4-SHA or AES256-SHA. You are wrong.
Edit: beachaccount: Financial institutions not using DHE is not a logically sound counter to "There's nothing wrong with using DHE algorithms, particularly if you're going to be transferring financial secrets around." While those institutions may chose not to, there is nothing wrong with others choosing otherwise. Additionally, patchy security on the part of one operator certainly is not a logically sound argument against this.
Furthermore, in the future, actually reply to a post in order to reply to a post. Not doing so unnecessarily confuses the flow of conversations (posts are not scarce resources).
The implication being that since they were sending the information in the clear in a separate part of their system, that they were wrong in configuring their https site as they did. I object to that conclusion.
Furthermore, one particular site using https in a particular mode when they don't need to use https at all is still not a logical argument against other sites using https with those ciphers.
https://www.ssllabs.com/ssldb/analyze.html?d=mail.google.com
haproxy direct: 6,000 requests per second
stunnel -> haproxy: 430 requests per second
(OLD) nginx (ssl) -> haproxy: 90 requests per second
nginx (AES256-SHA) -> haproxy: 1300 requests per second
nginx (AES256-SHA with keepalive 5 5;) -> haproxy: 4300 requests per second
Did other things change or is nginx more than twice as fast as the next best solution?