Hacker News Reader

Comment by ivoras | Hacker News Reader

Oh single points of failure, where art thou...

I hope everyone realises that Let's Encrypt is by now an essential part of the Internet, somewhat like DNS is, just massively centralised.

View on HN

If Let's Encrypt went away tomorrow, I'd Google "free SSL certificate" and find somebody else. Worst case, I'd either go back to the old way and buy a cheap certificate or go without SSL until a new free option comes along.

Honest question (I know just enough about SSL to be dangerous): Does certificate pinning throw a wrench in this plan?

If proper revoking procedures are being followed, no.

You can start reading more info here: https://developer.mozilla.org/en-US/docs/Web/Security/Certif...

(HPKP is considered obsolete, if by cert pinning you meant that. If you're confusing HSTS with HPKP, just know that HSTS makes it much harder to mistakenly access the site via http, while HPKP [now deprecated] is the practice of ensuring some hashes is found in the cert that the server sends to mitigate some attacks)

At this point, I don't think certificate pinning in the general internet/web environment is a thing.

Agreed. I MITM all my own traffic and I only have to exclude a handful of domains or A records on domains to not MITM.

Honestly curious, why do you MITM all your own traffic? For detailed logging?

Logging, ACL's for some mime types, overriding cache controls, shared cache for multiple devices, blocking some sites that I can't block using DNS.

> shared cache for multiple devices

Oh, that’s a neat one and would be very valuable for me. We have poor internet access at home and would be cool to reduce traffic going out to the net.

Thanks for the reply, appreciate it!

No problem. You can find examples of how to set up Squid-SSL-Bump or I can provide examples if you can't find any.

Before choosing to use pinning you should have planned for this situation. Rather than write it out here, I'll link a post I wrote in a Let's Encrypt forum in response to someone who'd just blown up their system due to a different pinning mistake.

https://community.letsencrypt.org/t/certificate-failure-due-...

Please don't enforce certificate pinning in user-facing software, at least without an opt-out. It becomes impossible to inspect my own traffic.

It also leads to a ton of clients who freak out when their app suddenly goes offline because they forgot to update the pinned cert. Seen that one countless times.

Pinning has uses, but if I'm running an app, I'm not doing pinning unless I have to.

Pinning to the issuing or root is much safer from an availability standpoint.

The hits for "free SSL certificate" certainly won't be able to issue 200M certs in a day.

AWS ACM is free-as-in-complimentary-peanuts, and I'd bet they could handle a pretty big chunk, although 2000rps for a request that's calling external services and cryptographically signing things is a bit intimidating.

That's only viable if you're using AWS's load balancers or CDN, though.

You can also use ACM certificates directly on nitro-powered EC2 instances too

https://docs.aws.amazon.com/enclaves/latest/user/nitro-encla...

Just manually encrypt everything and you're good lol

The TLS CA system is one of those places where a single failure point is absolutely better than distributed possibilities. That is the case because any failure anywhere compromises the entire system.

Of course, the other failure points didn't completely go away yet. But I do expect their number to reduce a lot in the future.

But it's a single point of failure that has about 60 days to come back online (if all actors are working fairly normally) from a full crash.

Quite. I'd hate to test it but 60 days is long enough to deploy a new CA and push the cert out to trust stores.

In an emergency.

It would take quite some coordination but I suspect the current pandemic might provide some models and examples as to the way to do things at scale with JiT decision making from wonks who normally drag feet by default.

They aren't the only no-cost distributor, their downtime doesn't really matter so long as it's less than a month long.

What exactly is the issue with centralization here?

> They aren't the only no-cost distributor

I couldn't name a second, which means they're probably not getting a huge %.

How would Netlify feel about not being able to issue or renew any certs for any sites for a month? Plenty of platforms rely on LE exclusively for one-click/automatic HTTPS for their customers sites.

Let's Encrypt isn't the only provider supporting ACME either. Sectigo (under the ZeroSSL name) is a notable alternative.

The thing is that we're not talking downtime here, but rather something in the CA compromised, which would mean that pretty much ANY website could be impersonated, as the attacker could issue a Let's Encrypt CA valid certificate for it. That is mitigated by invalidating this CA, but that also invalidates all legit certificates previously issued by them. So they need to reissue them all

Let's Encrypt publishes Certificate Transparency logs: https://letsencrypt.org/docs/ct-logs/

You can both block certs that do not appear in the logs, and decide which certs not to trust ("everything after Friday the 13th at midnight is not trusted"), once you know the date/time of the intrusion.

Chrome already blocks certs not appearing in CT logs, at least if it was issued in 2018 or newer.

hmm, the logs are valid point. what scenario are we addressing here then?

but the issuance time isn't relevant, they can easily backdate the cert

This is why they created the ACME protocol. There are actually other providers of this protocol now. So there is a possible future that is much more like DNS.

A lesson in recognizing, supporting, and defending public goods.

Agreed.

I'd love to see, say, another charitable CA set up in a different jurisdiction[0] that can hold a similar reputation to the ISRG.

[0] Probably somewhere in Europe, whether that's in the EU or elsewhere like Switzerland or the UK. What matters most is that it's somewhere with strong commitment to the Rule of Law and not beholden to the US.

Sounds like they have read-replicas, why can’t they fail over to a read replica as the new master?

Reply on news.ycombinator.com