How HTTPS Handshake Happens | Hacker News Reader

How HTTPS Handshake Happens

sudhakar.online

557 pointssudhakarrayav92 comments

As others have already pointed out, this explanation focuses on the RSA key exchange, which has been deprecated. It's not recommended for use with the current line of protocols (TLS 1.2 and earlier) and it's been completely removed from TLS 1.3 (work in progress, but close to being finished). The key weakness of the RSA key exchange is that session encryption keys are transported over the network encrypted with the server's public key. When the server's private key is compromised in some way, the attacker can passively decrypt all traffic, they can also "go back in time" to decrypt all previous conversations they might have recorded and kept.

For a more secure handshake approach, consider the Diffie-Hellman key exchange, which is a way for two parties to agree on a secret number (a very large number when used for encryption, of course) that third party can't discover from looking at the network traffic. It's a very simple and intuitive algorithm; there's a great video that explains it here: https://www.youtube.com/watch?v=YEBfamv-_do Today, we generally use the ephemeral elliptic curve DH variant (ECDHE), which has better security characteristics and works much faster.

However, DH is not sufficient, you must also authenticate the other party. Otherwise you could be negotiating your key exchange with the attacker and not the real recipient. This is where public cryptography comes in. After the key exchange, both parties use their private keys to sign the transcripts of the entire conversation up until that point. When this is verified you know that (1) you're talking to the right person and (2) that you have a secret (session keys) that you can use to efficiently protect the conversation.

Disclosure: I am the author of Bulletproof SSL and TLS, already mentioned in another comment here. Feel free to ask me anything.

Note that in most browser to web server communications that both sides don't usually sign the conversation during authentication, only the server does. This means the client knows it's the right server but the server doesn't know who the client is, which is why you have to use a username/password in some crappy web form for authentication.

Right. As you say, on most web sites the client is anonymous as far as TLS is concerned, and the server proves its right to respond to the requested hostname with a CA-backed certificate (and proof that it holds the private key that corresponds to the public key embedded in the certificate).

To drill deeper in the the handshake, however, both client and server do sign the initial handshake (with the final negotiated session secret, effectively). The idea is that you need to ensure that the handshake hasn't been tampered with. An active network attacker could have changed the original TLS connection request (ClientHello) to force usage of a particular cryptographic component they know how to break. For example, SSL v2 had no handshake protection which is why someone in control of the network could always enforce the weakest encryption (only 40 bits).

Ivan, I bought your book "Bulletproof SSL and TLS". It's excellent, and I recommend it to those who are charged with building, deploying, or managing a real world deployment that includes TLS.

I have yet to buy it, but I've recommended it to five or six people who said they wanted to understand TLS, because I've never heard of someone being disappointed by it!

I think I'll buy my own copy today. :-)

As an update, I've ordered a copy and look forward to reading it.

Would you recommend your book for someone with a rough understanding of how this works, but very light on detail?

In short, yes! But just to be 100% sure, I would encourage you to read the detailed table of contents (it's can be downloaded along the free first chapter from the book's homepage, in the right column: https://www.feistyduck.com/books/bulletproof-ssl-and-tls/ ).

One important point I would make is that my book doesn't dissect the protocols in detail. There is only one chapter (about 50 pages) dedicated to the protocol and I wanted to cover it at a level that virtually everyone will want to understand. If you want more than that, hundreds of pages won't be enough; in the end you will have to dive into the dozens of RFCs, and so on. What I do do, is point to those RFCs so that you at least know where to find the additional documentation. IIRC there are more than 600 links in the book and they can all be found here (automatically extracted from the DocBook XML manuscript): https://fsty.uk/bulletproof-ssl-and-tls

My realisation was that, in practice, actual protocol flaws are not the weakest link. Instead, there are hundreds of different problems everywhere in the ecosystem, so you need to look at the issues in the protocol, but also in the certification authorities and the certificate issuance process, SSL/TLS stacks, clients, browsers, servers, libraries, development and configuration practices, and so on. My book is thus very practical and examines all these aspects that you need to understand at at least some level if you want to use TLS.

BTW, please note that I will send you a free digital copy of the book if you buy the paperback _anywhere_ and send me the receipt. The benefit of the digital edition is that it's typically much more current than the paperback. In fact, the digital edition is fully up to date at the moment. I published a complete revision just yesterday.

> After the key exchange, both parties use their private keys to sign the transcripts of the entire conversation up until that point.

I don't see how this proves anything as a MITM attack would be able to do this just fine. Perhaps you mean this is where the CA-Cert comes in as the client can then verify that the process has not be tampered as the attacker can not sign with the matching CA-cert?

No, the attacker can't sign anything using either client's or server's private keys. The process assumes these keys haven't been compromised. But, as a client, you do need to be able to reliably verify that the private key used for the signing genuinely belongs to the server. That's where certificates come in. In the certificate there is a public key. The server has a matching private key. As part of the handshake, the client verifies that the two match. That's why the MITM can't interfere.

But that's only where the chain of trust begins. As the next step, the client must now verify that the certificate had been issued by a CA they trust, and they do that by recursively verifying the signature of each issuer in the certificate chain (along a few other things) until they reach a CA that's trusted by default (because it's in the client's root store).

Just to complicate things further: TLS 1.2 and earlier support many key exchange options, which implement key agreement and authentication in potentially different ways. For example, some key exchanges don't use certificates but rely on pre-shared keys. Another example, in the RSA key exchange the server proves ownership of the private key by being able to successfully handshake, and so on. TLS 1.3 made some improvements in this area and the key exchange process has been streamlined. My point here is that there are many other details to take into account, but I am trying to keep my explanations at a higher level.

As someone who's help create a certificate authority and helps people get certificates from it every day, I'm kind of amazed at how few people who are getting certificates know what they're for or could explain what they do, even at the level of "certificates protect site visitors against man-in-the-middle attacks". (Or even know that they possess private keys, which they've promised in their subscriber agreements to safeguard.)

Ivan - I have your book, I've read it (at least a year ago, if not more) - so sorry if this is covered and/or I should know the answers, but:

(1) Seems like the argument against RSA is "what happens when the server's private key is leaked?" But with ephemeral DH, it also requires private keys to be protected (now 2 of them!), these being signing keys. So isn't it simply trading one problem for another? Or perhaps the difference is that at least with DH, in theory only /new/ sessions can be attacked (MitM impersonating going forward), but older sessions can't be opened up.

(2) Doesn't this mean that now the client needs a keypair & certificate? Otherwise, how does the client sign? And where does the client key pair come from? I'm pretty sure my browsers (i.e., clients) don't have their own keys & certificates.

TIA for answering my silly questions.

(1) Yes, that's the argument against the RSA key exchange. I can break into your server, bribe or blackmail your system administrators, or take you to court to get your private key. This last case happened to Lavabit, for example.

Handshake integrity validation is always done, no matter which key exchange is used, and with effectively throw-away keys. So there's nothing there to protect long-term; those keys are going to disappear after the TLS session is complete.

The key difference with the RSA key exchange is that it forever leaves behind a piece of data encrypted with the private key (that can be recovered). Other key exchange methods use signatures, which have no value once the TLS session is complete.

(2) Public cryptography is not well suited for dealing with high volumes of data. Abstracting a little bit, handshake integrity is done via a HMAC, using keys that are negotiated for each session separately. Once the session is complete, these keys also disappear.

Hey Ivan, thanks for sharing my video :)

Hey, thanks for creating it. It's fantastic!

Can you suggest any compelling resource to encourage people to switch away from RSA key exchanges in their server configurations, assuming that they're not asking us? (When people ask me, I point them to the Mozilla configuration generator at https://mozilla.github.io/server-side-tls/ssl-config-generat...)

I just noticed two days ago that my bank, for example, uses an RSA key exchange and I was kind of dismayed about the non-forward-secret sessions. But I don't know what I could tell them as a customer that would make them care about this problem. (I don't think my theoretical knowledge of why their key exchange method is bad will really help, compared to some kind of industry or governmental guidance that says "this technology is obsolete and everyone should stop using it", or, even better, "this technology is obsolete and banks should stop using it".)

I think SSL Labs could be an option, but its penalty for configurations that don't support forward secrecy is not strong enough. That's my fault (sorry!), but I'll hopefully fix it soon.

Ultimately, I think the only way to kill the RSA key exchange is via a protocol revision, which is exactly what they've done in TLS 1.3. Sure, it will take a few years, but it will happen eventually.

I think banks like the RSA key exchange because it allows them to deploy transparent TLS traffic decryption via private key sharing.

> I think SSL Labs could be an option, but its penalty for configurations that don't support forward secrecy is not strong enough.

It's amazing how strong a motivation a grade or ranking can sometimes be. We saw this a lot with EFF's privacy practices report

https://www.eff.org/who-has-your-back-government-data-reques...

where companies were sometimes willing to make real changes to earn an extra star.

> I think banks like the RSA key exchange because it allows them to deploy transparent TLS traffic decryption via private key sharing.

Oh yeah, that was something they complained about a lot with mandatory PFS in the TLSWG. :-(

Why do they need this for their consumer-facing sites, though? The theory over in the working group was about monitoring activities of bank employees on third-party web sites (I thought), rather than about monitoring activities of bank customers on the bank's own web site.

curious, do you know where I can find out how proxy servers negotiate/relay SSL without eavesdropping? I'm trying to figure out how secure proxies communicate but can't find details :)

Well, the default is that TLS traffic cannot be decrypted or interfered with without detection, so no eavesdropping. A proxy can relay traffic at the TCP layer, but it won't be able to see any of the information (except for the parts that are not encrypted).

If you do want to eavesdrop, you have several options. For one, you can terminate the TLS traffic, access it, then create another entirely separate TLS session to send it elsewhere. This is easier with a reverse proxy, in which case the proxy is the server. If you want to transparently intercept client-initiated sessions you have to install your root CA key in each client device.

Going back to servers, another option is private RSA key sharing, which allows for passive decryption, but works only with the RSA key exchange. Finally, you can also play with session ticket keys. If you share those, and enable session tickets, you'll also be able to passively decrypt. This last option works even with DHE/ECDHE key exchanges.

thanks, i don't want to eavesdrop, but looking for technical docs on how a https proxy server really works :) .. something i can use to learn how to implement one.

How much lag and delays do SSL handshakes take and what are some ways to remain secure while minimising latency.

Technically, a full handshake adds 2 RTTs, and it's only one RTT on a successful resumption. In practice, I think that most modern browsers currently use something called False Start, where they cheat and send some data early, so even the full handshake takes only one RTT. False Start works only with a well-configured server.

In TLS 1.3, the full handshake is one RTT without any tricks; resumption also. There is also a 0-RTT resumption, but it sacrifices security for speed. Lots of people are unhappy about it.

If you are interested in details of the TLS protocol, check out these two books:

- Implementing SSL / TLS Using Cryptography and PKI [1]

- Bulletproof SSL and TLS: Understanding and Deploying SSL/TLS and PKI to Secure Servers and Web Applications [2]

In the first one the author implements the protocol (RSA/DH) from scratch (without even using any crypto library). The second one is a classic and contains a lot of interesting scripts (the chapter on using OpenSSL and creating your own PKI is available for free: https://www.feistyduck.com/books/openssl-cookbook/).

I spent some time studying TLS and wrote two blog posts [3][4], in which I decrypt the network traces of the TLS sessions. Maybe someone will find them interesting too.

[1] https://www.amazon.com/Implementing-SSL-TLS-Using-Cryptograp...

[2] https://www.amazon.com/gp/product/1907117040

[3] https://lowleveldesign.org/2016/03/09/manually-decrypting-ht...

[4] https://lowleveldesign.org/2016/05/10/tls-1-2-aes-gcm-and-ne...

I haven't read the books, and I'm sure that they are fascinating, but I am wary of any attempt to home-brew crypto. I'm specifically worried that some corner-cutters might use the Implementing SSL book's code or ideas in production.

I'm also against home-brew crypto, but the author clearly states that what he presents has only educational purpose. It's a bit like cryptopals - you also implement your own crypto in order to learn something.

I don't think that's the point of the book. There is enormous benefit that can be gleaned from building a "workable" implementation from first principles, even if it's not actually production "workable". These types of books are rare and very challenging to write, will definitely be taking a look.

What's the alternative, lock all books related to cryptography lest someone might do something in production? Educating is fine, and people reading relatively advanced technical books like this one should be considered as responsible, I think.

Not only reasonable, but it is a must. I think the "don't roll your own crypto" phrase is widely misunderstood. A better phrase would be - learn as much about crypto as you possibly can, to clearly understand why it is a bad idea to roll your own in production. This also means that after you learn it, you will have a much better idea which library, cypher or mode to pick for a particular task, and not just think "oh, crypto is complicated, i will just wrap everything in TLS and won't bother to check the certs"

People who are wanting to get into complex cryptography today have to start somewhere. Showing working models, with huge disclaimers, in blog posts and on github is a great way to lean, if you invite others to show you flaws and break your implementation.

If you want to build complex cryptography, you must either obtain formal training, or learn how to break complex cryptography.

What you can't do is read a book or two about cryptography, the way you would a new programming language, and then bootstrap your own expertise from that. Cryptography is a lot trickier than most other topics in computer science.

There is enormous difference between rolling your own crypto primitives and using existing primitives to create a protocol.

No, there isn't. Most crypto vulnerabilities happen in the joinery between primitives. If you rolled your own block cipher, it's less likely that the cipher would be broken than that your protocol would be broken with a vulnerability that would apply equally well to AES.

Obviously, don't make your own block cipher, either.

> Cryptography is a lot trickier than most other topics in computer science.

Is that actually true? Or is it that most software development isn't really about topics in computer science. Like if you wanted to design your own distributed database wouldn't most of what you said above also apply?

Isn't this really a case where "hard topic is hard"?

I thought about that when I wrote that comment and my conclusion is, no: distributed systems protocols are notoriously hard, but they don't have adversaries. Crypto protocol flaws are marginally more complicated than distributed systems flaws, but also, a distributed system with a byzantine fault can be survivable, and a cryptosystem with a subtle flaw won't be.

I'd buy that having adversaries is what makes crypto different than other topics. Thats probably a better way to pitch your message as well.

It's not "home-brew crypto" if it is a well established protocol.

Nonetheless, even an implementation of a well-established protocol could have problems like a timing attack or an oracle attack, so it's worth being extremely careful.

So a book that teaches a programming language with a "toy" software project does more harm than good? This is industry standard stuff.

Thank you so much for those book suggestions. They will either be exactly what I wanted to know or a great reference down the road.

There are some really cool "tricks" for avoiding the round trip — round trips are why everyone should be using a global load balancer for SSL. Clients have to send 2 packets across the world and wait for a reply, which can add >100ms before any actual work happens.

http2 helps because you can multiplex a bunch of requests into a single connection, less waiting on new connections to be established.

TLS 1.2 with session resumption lets clients reuse existing sessions after the first connection.

TLS 1.3 has a 0rtt handshake, which is pretty baller. It's just not widely deployed.

0rtt is only for resuming sessions. I believe the first handshake won't be any faster with 1.3. Still neat.

TLS1.3 does speed up non-0RTT handshakes too, at least the ones that use forward secrecy; which is nearly all these days. It reduces them to 1RTT instead of the 2RTT we have today.

Any idea when TLS 1.3 will be supported in NGINX? What browsers support it as well?

The problem is not in Nginx part, you must wait for TLS 1.3 implementation in OpenSSL, LibreSSL, etc.

It already is, if you build 1.13 with the right OpenSSL.

https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_...

https://github.com/tlswg/tls13-spec/wiki/Implementations

    TLS 1.3 has a 0rtt handshake, which is pretty baller.
    It's just not widely deployed.

And it likely won't be. 0RTT Allows for replay attacks (I capture your packets, and replay them). Without a round trip this will always exist.

Also 0RTT is only for re-connections not initial connections.

The solution is only like 0RTT be executed once and only once, but this isn't part of TLSv1.3 and is waiting to _globally_ approved. Until then nobody smart will touch it.

---

0RTT is only supported by Cloud-Flare and Google. Cloud-Flare makes you track 0RTT yourself so a fair number of sites are insecure. Google does SSL tick synchronization across data centers because their arrogant and is extremely vulnerable to timing att-

Wouldn't 0RTT be safe for GET requests that doesn't contain cookies or private data?

Granted I suppose such interpretation would have to be done by browsers.

And even then it could be used to figure out which site you are visiting.

You are assuming

* The TLS connection is instantly closed after the GET ends

* A new 0RTT ticket isn't created

* No session information is transmitted (most GET's even have session information for ads/metrics).

>"There are some really cool "tricks" for avoiding the round trip — round trips are why everyone should be using a global load balancer for SSL. Clients have to send 2 packets across the world and wait for a reply, which can add >100ms before any actual work happens."

What is a "global load balancer"? A load balancer doesn't avoid any round trips. The "work" of TLS begins as soon as the client sends a ClientHello which is during the second round trip. On a new connection the total round trips is 4 if you include the GET request. It's 3 round trips if you only consider the TCP hand shake and the TLS handshake. This is true whether there is a load balancer or not.

I assume a load balancing / caching solution that is available on an anycast IP address. The TLS termination happens at the (ideally) closest point of presence (PoP). The idea is to reduce the RTT from client to its termination point.

Think CloudFlare CDN or the Google Cloud Load Balancer.

Edit Mistyped RTT as TTL.

Sure, you can reduce the RTT by moving the edge closer to the eyeballs but that's not the same as avoiding an RTT as the OP stated. That's what all I was commenting on. There are mechanisms however to do that such as sessions tickets/resumption but that's not something specific to load balancers.

The OP didn't claim using a "global" LB would eliminate round-trips, just that you should use one because of the round trips.

The OP stated:

>"There are some really cool "tricks" for avoiding the round trip — round trips are why everyone should be using a global load balancer for SSL."

"Avoiding" means not incurring them, so yes they did claim a "global" LB would eliminate round trips.

That actually wasn't what I meant to claim, but it's too late to edit! It was an awkward sentence for sure.

The simpler sentence is: minimize round trip cost by getting close to users. Also avoid round trips if you can. :)

If you are terminating TLS on an edge network then you are likely caching the content there as well so as not to incur the latency in fetching it from origin. This is called a CDN not a load balancer.

There are a bunch of relatively pure global load balancer services available out there. Google Cloud's load balancer, for example. It works pretty well to terminate SSL close to visitors then maintain connections from load balancers back to app instances.

You avoid an RTT to the web server by having TLS terminate in the load balancer.

Why a load balancer and not a CDN? A CDN can definitely help.

Here's a good guide to an SSL handshake: https://www.incapsula.com/cdn-guide/cdn-and-ssl-tls.html#ssl...

The handshake adds 2+ additional round trips, so a CDN can definitely help this.

Agreed, I said the exact same thing in another comment on this thread.

It would be nice if the DNS lookup also provided the certificate for the site.

This is possible with TLSA records as specified by DANE.

Doesn't cover perfect forward secrecy (PFS).

Without PFS if the server's private key was stolen (e.g. by hacking the server) then all traffic sniffed in the past could be decrypted.

See: https://en.wikipedia.org/wiki/Forward_secrecy

Genuinely curious, as HTTPS is something I do not fully understand even with this simplification:

If the browser's symmetric key is encrypted with icicibank's public key, why can't a sniffer unlock it by also requesting icicibank's public key and decrypting the key sharing message?

Only the private key can decrypt it. This is the difference between asymmetric and symmetric encryption algorithms.

The payload is encrypted with the bank's public key but it can only be decrypted with their private key. This is the basis of public key cryptography [1].

[1] https://en.wikipedia.org/wiki/Public-key_cryptography

It's quite counterintuitive that this is possible at all. We didn't even know a way to make it work until the 1970s!

When talking HTTPS it's important to note that the protocol uses both asymmetric and symmetric key encryption.

Asymmetric key exchange for secure session setup / authentication

Symmetric key for secure session data encryption

https://tools.ietf.org/html/rfc4346#section-1

you can encrypt with public key, but it cant be decrypted with same key. its assymetric encryption.

Public Key encryption is like a bank deposit box.

Anyone can encryption (put things in), but only the Private Key (the bankers) can decrypt (see whats inside).

> If the browser's symmetric key is encrypted with icicibank's public key

For Diffie-Helman key exchange, atleast...

Encrypting/"locking" uses the public key.

Unlocking a public-key-encrypted-symmetric-key happens with icicibank's private key, which the sniffer does not have.

Here's another diagram: https://en.wikipedia.org/wiki/Diffie%E2%80%93Hellman_key_exc...

The content is actually pretty terrible and not likely to help anyone understand much about TLS.

This however is a decent resource for learning the things you're asking about such as public key crypto, asymetric vs symetric, digital signatures, etc. Specifically weeks 5 and 6. The whole course is good though.

https://www.coursera.org/learn/crypto#syllabus

Just to clarify other people's responses to make this ultra clear for people learning:

- stuff encrypted with the private key can only be decrypted using the public key.

- stuff encrypted with the public key can only be decrypted using the private key.

This is why the public key can be distributed around freely and anyone can use it to encrypt, because only the person with the private key can decrypt it (and this is why you really need to keep your private key secure!)

The flip side is anyone can decrypt something encrypted with the private key (since the public key is widely known). But this is how we can verify that the other server really is who they say they are - remember that the public key can only decrypt stuff encrypted with the private key, so if we can decrypt something from the server then we know that they encrypted it using the private key and are therefore probably legit (...or the private key got leaked or CIA/Snowdon etc)

Enjoy learning! :)