Milestone: 100M Certificates Issued
letsencrypt.org
letsencrypt.org
SSL certificate from a traditional provider valid for two years: $20.
Automated SSL certificate generation and deployment via LetsEncrypt with zero human intervention and more importantly zero human intervention to renew it going forward - priceless.
---
That's the real value for me. At $10/cert, that's not even a rounding error. But manually generating a new CSR, uploading it via crappy web form, waiting a random amount of time, proving domain ownership by responding to an email (sent in plaintext), waiting a different random amount of time, downloading the new cert (again usually sent via plaintext email), and finally copying it over the old cert and reloading the SSL conf ... now that costs some serious time and time is money.
Right now, I have LE certs for my main website and for a test webserver hosted on my laptop, both renewed automatically and painlessly through cron jobs. The fire and forget nature of LE is very convenient.
EV guidelines have nothing to do with Let's Encrypt or ACME. They state that even for renewal all checks have to be done again, which cannot be automated.
For the Domain Name Validation (which is the main thing web browsers and several other kinds of software care about) this is done by one of the 10 Blessed Methods. ACME automates three of those ten methods. Some others can't be automated (e.g. get a bona fide lawyer to sign a physical paper document saying they've checked you really own the DNS name)
EV requires also validating the Subject entity's name and place of business. This cannot be entirely automated today. But some CA issuers specialising in EV get very close, like an hour turnaround in most cases for businesses in places where they operate.
The intent in ACME is that semi-automated processes would be handled to one side of ACME, you'd do them offline, and you'd gain "authz" which are how ACME tracks validation, for the things which had been validated offline.
Also, it's not necessary to re-validate everything for every new issuance. Suppose you prove you're really EXA Metal Pole Location Executives, business #123123123 of Some City, Texas. A CA would be entitled to rely on having validated that for some time, I think it's currently up to three years. In that time you could issue and re-issue hundreds, even thousands of certificates with no extra fuss.
Also keep in mind that if you're a big customer the big CAs already had products where things are streamlined for you, so the annoying experience you get as a small operator is not necessarily happening at scale for a large corporation like, say, Disney.
The "old" way required all customers/domain owners to either give us control of their domain or to answer an email every time we needed to get a new cert. Now we've automated moving around domains between certs and generate new ones as needed. Validation by email to 100 different people is like herding cats. Validation that happens in an automated way - priceless.
I'm not saying you should design this way from the start, but this is what we grew into and SAN works for us. I only wish the limit of 100 per cert was higher.
You then have to wait for them to respond back to you and you'll mostly get a reply with something like "I thought our emails were with you" or "We don't know who our provider is". Like, you're paying someone for some service, how can you not know who is providing you that service?
Then when they do finally set the email up, you have to go through the process which you just spoke about. It's such a pain, but with LE, it cuts all of this crap out and makes for happier businesses.
Given that traditional CA models don't actually have much more security than LE (in fact from personal experience they're far less secure), I wonder whether they should be in the business of providing wildcard certificates at all.
There are vanishingly few reasons to ever share private keys across multiple automated systems. This is not one of them. (Are your systems not automated? Fix that.)
However, I would also point out that wildcards allow people who aren't you to create domains that you didn't create and represent them as being from you, which is an attack vector that even SAN enumeration doesn't allow. We're talking whitelists over blacklists.
It's not about if I'm using SANs anyway. It's that LE is fine with that, but not with wildcard certs.
> However, I would also point out that wildcards allow people who aren't you to create domains that you didn't create and represent them as being from you, which is an attack vector that even SAN enumeration doesn't allow. We're talking whitelists over blacklists.
Can you elaborate what you mean here?
> Can you elaborate what you mean here?
Me saying "this cert is valid for example.com, foo.example.com, and bar.example.com" is an incrementally better security posture than saying "this cert is valid for *.example.com". It constrains an attacker from running a phishing attack from, say, "private.example.com" or "secure.example.com". You're whitelisting what you're covering by explicitly declaring all of them.
It is incremental, but it's a thing.
When it comes to security, including phishing, it behooves all of us to not assume that permissiveness is a de facto positive.
Obviously this is highly dependent on the usage of the certs to begin with.
If you work through all of the ways ACME verification works (/.well-known/, DNS, and so on) you can come up with several plausible setups that become insecure if you allow wildcard certificates.
Here's a stack exchange question about it: https://security.stackexchange.com/questions/157972/why-cant....
> The authentication test can't merely back down one level to mysub.example.com because that isn't covered by the cert requested, and may not be under the control of the requesting agent either.
This just seems wrong. Performing the check like that seems to be a sane way to do it. If people can't control the base domain they have no business requesting a wildcard cert.
Alternatively they could just do a DNS check. If you can configure DNS for a.com, you control *.a.com.
"heavily implies" is too low of a bar to make it safe for a CA to provide wildcard certificates through an entirely automated process. While they could do it, they care enough about security to make the decision not to.
> If you can configure DNS for a.com, you control *.a.com.
This assumes that if you can create a TXT record for a.com that you must also have access to the ability to create other records for a.com.
While it is a stretch, I would say that the only bar sufficiently high would be fulfilled is forcing an administrator to prove that they can change the NS records for a domain. Which I don't believe would be a practical thing to test, and might run into problems with things like cloudflare.
Yup! And LE is totally fine with this! Have you read the ACME DNS verification spec?
The "grey padlock" for an otherwise perfectly valid certificate seems to confuse a very small subset of users, especially when there's a ton of badly written articles out there advising users only to trust "green padlocks", given the behavior in almost all other browsers. Thankfully showing google.com not using an EV certificate is usually easiest way to shut these conversations down.
It's not so much about browser-specific behavior, it's about using the padlock to ensure that the company or other entity you're interacting with is the one you wanted, as opposed to only ensuring that the server you're talking to is the one indicated in the address bar.
A gray padlock is better than none, but telling less technically-inclined, clicks-all-the-links, fat-fingered users to trust them outright may not be the best idea now that you can go get bankofomerica.com, dress it up to look like BoA's site, and put a gray padlock on it for free.
I think Chrome/Firefox got this one right by using a green padlock for both valid EV _and_ DV certificates like Letsencrypt, displaying the extra EV information clearly inline in the URL bar if it is present. The changing color in Safari/Edge can give the impression that the the non-EV certificate is somehow less secure, which in technical terms is not the case. Questions of identity verification and security of data transmission are two separate issues which the Safari/Edge approach arguably conflates - both EV and DV can provide equal security of transmission, even if only EV provides some degree of identity confirmation.
As a user, it's simply not possible to make the distinction too - if I visit your fake Bank Of America site with a grey padlock, how can I even be sure the 'real' site had an EV certificate and a green one, when there's so many companies that don't? As a user you are effectively forced to treat the grey and green padlocks the same, so why not make them the same color and display extra EV info if it is there, like the other browsers?
They could, so they did.
"Why pay for something that’s free, you ask? When a user visits a website that is secured with a self-signed, or free SSL certificate, most web browsers will post an error message. While some people will click “I understand the risks” and proceed to your site, it’s likely that many people will click the “Get me out of here!” button and never return.
The real problem lies in the fact that self-signed certificates are virtually unregulated. If your site is compromised, it may still appear secure; however, certificates issued by a trusted certificate authority can be revoked and therefore alert users of any potential threats."
Source: http://searchengineland.com/comprehensive-guide-ssl-certific...
That article makes the mistake of assuming that if a certificate is free then it _must_ be self-signed, which is obviously false.
BUT, being _able_ to make certificates is very expensive. ISRG (the charity behind Let's Encrypt) spends _millions of dollars per year_ on this.
Purely mechanically Let's Encrypt needs a data centre with a bunch of high availability servers and pretty good Internet access, employees to look after all that stuff, a deal with CDNs to serve up their OCSP responses around the world (which ensure certificates stay working after they're issued) and so on, none of which is cheap and all of which needed to be in place before issuing even one certificate.
In practice to be any use the certificates need to be trusted, which meant meeting the standards of the big Trust Stores (definitely Apple, Microsoft, Mozilla, probably Google, Oracle, and ideally dozens more besides). Those standards require physical site security, regular third party audits, and so on, which cost even _more_.
Now, for Let's Encrypt the solution is this is a charity, you can pitch in $10 and if a hundreds of thousands of people like you do that then it all gets paid for, or if a few bigger organisations pitch in $1000 each that helps too.
But for a commercial outfit, well, why not charge $10 to cover costs? And if $10, why not $50? Why not $1000?
That also means one can make effectual suggestions to webmasters.
I told a webmaster who maintains a forum about LetsEncrypt a few months ago and that I'd help them set it up just so that users aren't practicing the bad habit of logging in over http. I just went back to check and the forum is now https by default using LetsEncrypt certificate.
I couldn't have achieved that a few years ago. The pain of setting up SSL would have inescapably digressed into a low-quality flamewar about how PKI is broken.
For businesses, sure, but the fact that you had to pay money at all was very discouraging for personal websites!
Unless you are on a shared hosting, which is still one of the most popular options.
Particularly, Google and Akamai...two of the biggest LE sponsors. They both retain good visibility to user behavior (like specific urls visited) because of things like GA,MITM proxying, etc. But, ubiquitous availability of that is taken away from ISP operators.
Which is a good thing. Makes me curious if there's anything else like this that could be achieved. Are there other net public good projects that align well with deep pocket potential sponsors?
Also security professional is a person that know a lot about security, his intentions are not necessarily "good" for him to be considered a security professional in my opinion.
Very good black hat hacker are also professionals.
It doesn't happen in a vacuum.
Letsencrypt is cheap (free) and easy to use.
Even people with not a lot experience can secure their sites and apps, and it just works. Yes, you have to update it every three months, but that's worth the price and the excellent documentation.
Before letsencrypt I always wanted to secure my blog with https but never got around to it or it just looked super complicated and error prone.
Then, my provider built its own tool for LE and its just so easy to implement.
This is just a cronjob, no?
* Security in case of a breach
* Ephemeral containers
I try never to store secrets in an unencrypted cloud filesystem, and decrypted secrets should be in memory, if possible.
I store my LE keys in Vault using ten-ply-crest, so access is very restricted yet storage has redundancy. Vault is dead easy to integrate with any system.
I have been really surprised how many LetsEncrypt tools store certs and keys in the filesystem, other than "that's the way it's always been done".
Also if the keys are stored only on one cloud server, that means there ought to be a backup somewhere, so now there are two potential leaks instead of one.
With Vault only ten-ply-crest and Fabio access the keys. No human ever gets near them.
I also use Subsonic @home which requires generating a keystore as the app doesn't yet handle LE natively.
With certbot it's actually easier to update every three months with Letsencrypt that updating every/other year with a traditional SSL CA.
I don't have to update it at all. A cronjob does that for me now.
It can be automated. On mine, cronjob does the job!
I understand that the problem they solve: A user wants to get the public key for a certain domain. So he knows he is talking to a server by the domain owner and not some man in the middle.
So he asks a third party whos public key he already has. In this case Let's Encrypt. Ok.
But how did Let's Encrypt get the public key from the domain owner?
I know they make the domain owner install some software on his server. But how does that make sure they talk to a server by the domain owner and not a man in the middle? Does the software include some private key so they can send them a "hey, encrypt this" message and to prove the server in fact has that private key?
If so, why is the Let's Encrypt software so complicated and not just a 5 line script or something?
You claim to have domain x.com. LetsEncrypt sends you a token. Place it in a DNS record or in a pre-determined path on your domain and ask LetsEncrypt to read it back.
If it matches, it is verified that you own the domain.
How is the scheme you propose simpler, whilst retaining the same base guarantee (that the requester is the owner of the domain in question)?
That would be easy and elegant. But they want you to install software on your server.
You don't necessarily have to do any of the above on the server for which you want the certificate. You can do it somewhere else and then transfer the resulting certificate to the intended host (either manually or with a shell script/ansible/puppet whatever).
But you can place these tokens on your webserver by hand if you want, in a pre-determined path.
Verifying domain ownership is what the challenges are responsible for.
The http or dns challenge/response is more reliable and pretty easy to automate and scale to many domains.
We'd have to go back to physical snail mail using the address on the domain record, and/or notarization. Both of which of non-automated and very slow (relatively speaking).
- LE makes a request to example.com, to a known URL that contains a key that LE told you to put there
- LE does a DNS lookup for a specific TXT record (again, containing a key they told you to put there).
The complete answer to your question can be found in the ACME spec (the protocol LE uses):
https://ietf-wg-acme.github.io/acme/draft-ietf-acme-acme.htm...
They might have "a few places around the internet" to verify from, but I doubt it is very diverse location-wise. And there's no way they're going to bother with doing it multiple times, for free.
You are aware you are talking about the cost of a single HTTP GET?
First, the conversation is about the possibility of multiple (not single) interactions, including protocols other than HTTP such as DNS.
Perhaps you're confusing a subset of what they currently do vs. coordinating the initiation of requests multiple times from a few places around the internet (per OP) supporting multiple protocols, aggregating and verifying the responses, etc. -- all within a reasonable timeframe at scale with Internet infrastructure level high availability (while protecting against malicious misuse)?
I hope this is enough to explain that no, that's not what I was talking about. But thanks for taking the time to reply!
Of course it doesn't prevent a total hack of your infrastructure but when you've come to that point, you've got bigger issues.
> So he asks a third party whos public key he already has. In this case Let's Encrypt. Ok.
It works quite differently -- the browser doesn't have to talk to Let's Encrypt at all to verify correctness.
How it works:
1. Browser connects to the server
2. TLS handshake, crypto-stuff
3. The server cryptographically presents the browser his certificate
4. The server's certificate is the server's public key, some attributes (like what domain names the cert is valid for) and a digital signature of all that stuff
5. The browser has a list of certificate authorities
6. The browser checks whether the digital signature on the server's certificate is valid and was made by a CA the browser trusts
7. The browser checks that the certificate is valid for the site it's currently accessing
8. Connection legit
The problem that the CAs (including Let's Encrypt) solve is not distribution of public keys, but rather trusting public keys presented by someone.
> But how did Let's Encrypt get the public key from the domain owner? > I know they make the domain owner install some software on his server
What the ACME clients do is two-fold:
1. They put up some sort of challenge response mechanism. This is used by Let's Encrypt to test whether a client is authorized to rule over a domain.
2. They generate (or use) a "Certificate Signing Request" (CSR). A CSR is basically a digital "sign here" field. The CSR contains the server's public key, what domains it wants to represent etc.. The CA checks the CSR using the challenge-response mechanism from (1) and then signs it digitally. The result is a certificate, which is sent back to the client and installed by the client. The private key of the certificate never left the server.
> If so, why is the Let's Encrypt software so complicated and not just a 5 line script or something?
Well it doesn't have to be complicated. acme.sh is only five (thousand) lines of shell :)
There are a bunch of ways the challenge-response mechanism can work, and there are also a lot of other options, which add to the complexity, as well as installation routines for certificates, which support a bunch of different servers.
A specific case with "static" challenge-response could conceivably boil down to just a dozen lines of shell or so.
The CSR is a signed data structure that says "Hi, here's my public key K1, and I want you to issue me a certificate with properties P1, P2, P3 for names N1, N2, N3" and it's signed using a private key K2.
Using K1 (which is right there in the document) we can check that the K2 signature matches. If not, we can't be sure the real owner of K2 even wanted this certificate and we must reject it.
Then we can look at the list of P1... and N1... and decide if we want to issue a certificate at all. If we do, we can choose to issue any certificate we want, we don't need to issue exactly the one described in the CSR.
This is important because many CSRs, whether generated by well-meaning admins or automatically by an appliance or service, are full of crap that Let's Encrypt doesn't want to, or isn't permitted by the Baseline Requirements to sign.
[ e.g. it's very common for CSRs to claim that London is in a County called Middlesex. Nope, that county was abolished decades ago, London isn't in a county at all, but people see a field labelled "State or County" and they figure they should write something in the box. If a CA signs that, they're claiming Middlesex still exists. ]
By constructing each certificate from first principles after examining the CSR, Let's Encrypt can ensure they sign only certificates for exactly the things they've validated, nothing else.
As our community project, we built a totally free to the public service to monitor certs and alert you when they get close to expiration or are invalid, etc:
Feel free to use it for any certs.
I'm saying, you're putting it in the wrong place. It's not a monitoring problem, it's a configuration management problem--"I have defined a policy, am I in compliance with that policy?". And, from experience consulting with a pretty wide range of clients, putting it in the monitoring stack encourages manual work instead of automation. Instead of having active and self-correcting systems, you create passive and manually reactive ones. Monitoring is for stuff you have to react to. Automation and automation testing is for stuff you should never be reacting to.
There are plenty of other things related to your TLS certificate you might want to monitor that don't quite fit into CM (OCSP revocation status and availability, unexpected key rotations, etc.), so you'll likely need separate monitoring either way.
I've been using another free certificate monitor, but it assumes 1 year validity and the lowest possible warning threshold is 30 days, which isn't all that useful when working with Let's Encrypt.
So, you have another happy user now :-)
While registering, I noticed timezone as "Indian Time". It should be "Indian Standard Time(IST)". </nitpick>
Not that all of them should survive, there are a lot of crappy services that deserved this. But I'm just trying to place myself in their CEOs position and wondering how the game plan should be.
My guess is Heartbleed made him re-consider whether he wanted to keep doing ssl/tls certs, and once Let's Encrypt came along he started looking for a way to get out of it.
This way you can run multiple load-balancers and keep your existing infrastructure while still enjoying all the benefits of LE. Written in JavaScript but works with any stack.
Based on the great rawacme, and we'll keep this updated as ACME hits 2.0! [2]
[0]: https://github.com/nextorigin/ten-ply-crest
I worry that at the moment there are probably a lot of systems out there that leave DNS API keys lying around on endpoints because it's easier to automate the renewal than write a convoluted two stage ansible role/play that delegates sensitive actions to the controller running the play.
There's a neat trick with CNAMEs and a separate DNS zone that works kind of as a workaround, if you're okay with introducing another piece of infrastructure[1].
Will check out the CNAME trick - thanks!
I have an industrial VPN with ~300 embedded devices behind them. For some reason the user wants to access to each one of it via https.
I thus set up an Apache proxy which proxies https://device_n.vpn.cust.tld to the VPN device (and some auth)
LE cant issue 300 certificates and it would be very ugly.
For systems not explicitly public, whitelists are almost always better than blacklists.
The limit is 100, unless they removed that when I wasn't looking. Back in the beta I tried one domain, a handful and then, because I couldn't find a technical reason why SAN should be limited, I tried 500. It fails. I have many certs with up to 100 and would be happy if this changed.
(And if so, in the prior commenter's situation--okay, create three certs. LE might not support 300 certs, but 3 with 100 SANs should work just fine.)
I know the names, but rerolling a cert takes time, takes up resources, etc.
As I said in my post at
https://www.eff.org/deeplinks/2017/06/lets-encrypt-has-issue...
it's probably between 17 million and 46 million web sites, although it could be a bit smaller than 17 million at the most restrictive notion of "web site" if you don't consider subdomains to be a separate web site even if they have different content and are operated by a different person.
You can see all of the certificates at
https://crt.sh/?Identity=%25&iCAID=7395 https://crt.sh/?Identity=%25&iCAID=16418
and you can also get a lot of the data from
or https://www.censys.io/ (registration required), although the people running it told me in connection with this that their Certificate Transparency data is not as current at the moment as crt.sh's data.
https://github.com/JrCs/docker-letsencrypt-nginx-proxy-compa...
I must have at least 4 accounts there due to them using client certificate login. Great proof of concept why certificate based website logins never took off.
I started writing a small utility to do just that (and then upload the cert to CPanel using their API). It works, though I'm lazy to make it reliable enough for release.
The market is probably (a) people who want to support/donate to LetsEncrypt and (b) people who are ultra-paranoid / strongly prefer less moving parts in ops.
The one major downtime I'm aware of so far (a month ago) could also be a factor: Let's Encrypt is down | https://news.ycombinator.com/item?id=14374933
https://groups.google.com/a/chromium.org/forum/#!topic/blink...
https://groups.google.com/a/chromium.org/forum/m/#!msg/blink...
> As captured in Chrome’s Root Certificate Policy, root certificate authorities are expected to perform a number of critical functions commensurate with the trust granted to them. This includes properly ensuring that domain control validation is performed for server certificates, to audit logs frequently for evidence of unauthorized issuance, and to protect their infrastructure in order to minimize the ability for the issuance of fraudulent certs.
> On the basis of the details publicly provided by Symantec, we do not believe that they have properly upheld these principles, and as such, have created significant risk for Google Chrome users. Symantec allowed at least four parties access to their infrastructure in a way to cause certificate issuance, did not sufficiently oversee these capabilities as required and expected, and when presented with evidence of these organizations’ failure to abide to the appropriate standard of care, failed to disclose such information in a timely manner or to identify the significance of the issues reported to them.
i.e. they apparently weren't even checking requesters controlled the domain they requested a certificate for in some instances!! And that's pretty much the only requirement for DV certificates?
My point is that the legacy ssl providers you're referring to were doing less checks than Lets Encrypt!
[1]: https://groups.google.com/forum/#!msg/mozilla.dev.security.p...
[2]: https://groups.google.com/forum/#!msg/mozilla.dev.security.p...
fake-paypal.phish.com
As designed, it was supposed to be that:
Domain Validation (DV) certs would indicate that your connection was secure.
Extended Validation (EV) certs would indicate that your connection was secure and you were connecting to the proper site.
In reality, the problems with phishing and mis-issued DV certs got so bad that that issuers like GeoTrust (owned by Symantec) began keyword checking DV certs for major company trademarks, phishing targets, etc. that depending on what it was would put your through varying degrees of scrutiny.
Examples:
- Some used bookstore had a domain like "the-good-book.com" which got flagged for manual review b/c of the "book.com" string and so many "facebook.com" phishing attempts.
- Anytime people would request certs for BMW, HBO, would trigger requests for contact information within those organizations and often a phone call from GeoTrust
- It was basically impossible to get anything issued with the term "paypal", "bank" or "apple" in the domain.
Please note that these were all for run of the mill - $15 single domain, DV certs.
This was for a fairly high end service that required credit card accounts, etc and we still saw around 5% of requests get pulled (rightly or wrongly) into one of these manual review filters.
It was a fairly unsung and unappreciated part of the what the cert issuers were doing - but it absolutely 100% helped.
LetsEncrypt is forcing the Cert and Browser industry back to the original DV/EV split and it's unclear where we're going to land.
Right now even technical people _trust_ the green lock and it's wildly easy to miss the fact that you're on a domain that is paypal-login-security-usa-1.com
I think we're going to see a future where the browser indicators are:
Red -> No SSL
Gray -> DV SSL
Green -> EV SSL