Hacker News Reader

If you use caddy, you'll have other issues to think about anyway.

The DNS RFCs (and later the URL RFCs) standardize hostnames to be a set of labels, separated by ".", a relative hostname ending in a label, an absolute hostname ending in a dot.

If you are in a corporate system, your DNS resolver will try to append its domain to any relative hostname.

So in the domain kuschku.de, if I do ping lithium, it will resolve to lithium.kuschku.de, and use that IP address.

This feature is used for example by kubernetes' DNS resolver.

This also means that if I enter the domain caddyserver.com, and try to ping it, it will first resolve to caddyserver.com.kuschku.de (or caddyserver.com.svc.default.cluster.local).

Now, to avoid these unnecessary lookups, you can simply try to access caddyserver.com. — a valid domain, valid URL, and per RFC it should point at the original, outside of any corporate domain. Except, it doesn't work.

Nginx, Apache, IIS, as well as the GCP, Azure and AWS ingress system support absolute URLs. (And all major websites in the Alexa Top 1 million that don't use caddy or traefik also support this. As does news.ycombinator.com.)

Caddy does not support serving these (instead mholt recommends you just copy-paste every config rule you have twice), and the bug was closed as WONTFIX.

> randomly

What? How does that make any sense? Why wouldnt you just try all undetstood (and not manually disabled) challenges until you get a successful result?

This isnt like picking an ephemeral port number.

So an attack against one challenge doesn't deterministically succeed.

The much greater likely scenario is an error in comms/le (such as this issues today) means Caddy will un-deterministically fail

That is a trade-off for false negatives rather than false positives. It is not an unreasonable choice when dealing with certificates.

The default client recommended by LE (certbot) supports multi-challenge registration/renewal. I'm pretty sure I trust their judgement over that of the Caddy project, given their (Caddy) history of weird decisions that backfire and cause user issues.

In addition to what was already mentioned, it's also to be polite. Imagine if Caddy and other clients already did what you suggest: Let's Encrypt would suddenly be receiving ~50% more traffic during an already-stressful time. When we were first integrating ACME into Caddy, traffic spikes like this were a problem (for other reasons, long since fixed), which we now are very sensitive to avoid.

The default recommended client does support multi challenge, and from what the logs show, the ACME protocol even handles this without extra load - it returns the challenge modes it will accept.

I wasnt aware of this and you arent either apprently. The difference is, I'm not shipping a commercial web server package whose key feature is LetsEncrypt (and thus ACME) support.

Caddy's current implementation is more resilient than most other ACME clients in the event of an incident like the one Let's Encrypt is currently experiencing. Typical client implementations (such as certbot, but most others too) are configured to use exactly one challenge type at the time of setup and will continue to use this challenge type for renewal attempts, eventually failing to do so in time if the challenge type remains disabled for more than 30 days (or whatever schedule the client uses).

We can argue about the trade-offs of picking the challenge type at random versus, say, trying one and then falling back to another (both approaches have their pros and cons), but that's already more than what you'd typically get from ACME clients.

> Typical client implementations (such as certbot, but most others too) are configured to use exactly one challenge type at the time of setup

Certbot has a preferred challenges flag, which takes an ordered list. Setting it to tls-sni-01,http-01 means it will try to use the TLS challenge first, and fallback to the HTTP challenge. I've tried this myself in dry-run mode (because my existing certs all still have valid authorizations against the TLS-SNI challenge) and it will drop back to http-01.

It's definitely an option, but the question is whether someone who has set up certbot at some point in the past will benefit from it without manual intervention (as is the case for Caddy). If you're following the standard setup, using the instructions on certbot's website, you'll select exactly one plugin (with each plugin mapping to one challenge type, unless you do something fancy), and if that challenge type goes away, manual intervention is required.

> but the question is whether someone who has set up certbot at some point in the past will benefit from it without manual intervention (as is the case for Caddy)

This whole thread is responding to the author of Caddy saying specifically that people need to intervene or Caddy won't necessarily renew properly.

I specifically said in the above comment, that doing a renew in dry-run mode (because a --force-renewal re-uses the existing challenge authorisation my certs have against TLS-SNI) correctly falls back to doing the http challenge. The only "manual" part was that I ran the command - if the certs are due for renewal and the regular cron/systemd timer calls it, the TLS-SNI authorisation will either still be valid, and it will renew as expected, or it will need to re-authorise via challenge, and it will fall back to the http challenge as I just demonstrated, and renew as expected.

> and if that challenge type goes away, manual intervention is required

If all the methods you chose (e.g. TLS-SNI,HTTP) went away, sure, intervention would be required. But if they all went away, no client would work.

> If you're following the standard setup, using the instructions on certbot's website

If you follow the standard setup for Caddy you have a 50/50 chance of getting a cert today. Either you accept that people can deviate from the most basic "this is how you can run it as an example" or we stick to the defaults both ways. Claiming "oh but the standard setup.." when this issue requires caddy users to change their freaking init/sevice files is disingenuous.

For reference, I use HAProxy + Certbot and a tiny shell script - and it hasn't been susceptible to any of crazy issues Caddy has had just in the last 12 months, due to weird intentional behaviour of the program.

> This whole thread is responding to the author of Caddy saying specifically that people need to intervene or Caddy won't necessarily renew properly.

No, that's not what Matt said:

> Your sites will likely not go offline even if you do not use this flag because Caddy tries up to 2 times per day, 30 days out, to renew an expiring certificate, as long as you keep it running.

> the TLS-SNI authorisation will either still be valid, and it will renew as expected, or it will need to re-authorise via challenge, and it will fall back to the http challenge as I just demonstrated

This is not an accurate description of how you'd be affected with typical certbot deployments. Two examples:

Debian Stretch with apache2. Certbot's website recommends the following command:

    sudo certbot --apache

Currently, this fails with the following error message:

    Client with the currently selected authenticator does not support any combination of challenges that will satisfy the CA.

Same OS, nginx. Certbot recommends this command:

    sudo certbot --nginx

This causes the same error.

Renewal would fail with the same error message; You can find examples on Let's Encrypt's Community Forum if you don't believe me[1].

Is it possible to set up renewal in a way that would cause certbot to fall back to a working challenge? Sure. But it's far from common or even the default.

> If you follow the standard setup for Caddy you have a 50/50 chance of getting a cert today. Either you accept that people can deviate from the most basic "this is how you can run it as an example" or we stick to the defaults both ways. Claiming "oh but the standard setup.." when this issue requires caddy users to change their freaking init/sevice files is disingenuous.

Again, you're misinterpreting what Caddy does. Renewal will only fail if you're unlucky 60 times in a row, as opposed to 100% of the time for most clients in common scenarios.

[1]: https://community.letsencrypt.org/t/client-with-the-currentl...

> Is it possible to set up renewal in a way that would cause certbot to fall back to a working challenge? Sure. But it's far from common or even the default.

Is it possible to make Caddy deterministically use the capabilities of the ACME protocol to try the intersection of challenges you want to use, and the ACME server supports? No. It's not.

The rest is fucking moot. My point in every one of these threads is that Caddy is presented as "it just works" until it fucking doesn't and you have the weirdest shit happening.

With regular tools, it's expected that you have a clue what the fuck you're doing and configure it to meet your requirements.

Inexperienced people may use certbot and end up needing to do something. Sure. But I don't expect inexperienced people to be managing production environment servers. Do you?

You claim Caddy's defaults are better - thats an opinion, but the problem is it's not a "default" it's forced fucking behaviour. You simply can't tell it to operate sanely like you can with certbot + haproxy.

> Is it possible to make Caddy deterministically use the capabilities of the ACME protocol to try the intersection of challenges you want to use, and the ACME server supports? No. It's not.

Very, very few ACME clients have a renewal mechanism that supports automatic fallback to a different challenge type. I know a couple and the only one I can think of is certbot, and even there that's only true if you use the standalone or manual plugin (the others don't support "--preferred-challenges").

However, if your question is "Can I make Caddy use the exact challenge type I want it to", then the answer is yes, as you can guess from the "-disable-tls-sni-challenge" flag. This is what most clients will let you do. No one is saying that those clients can't renew with manual intervention.

> The rest is fucking moot. My point in every one of these threads is that Caddy is presented as "it just works" until it fucking doesn't and you have the weirdest shit happening.

You're changing goal posts.

> With regular tools, it's expected that you have a clue what the fuck you're doing and configure it to meet your requirements.

Part of knowing what you're doing as a developer is knowing that you either have sane defaults or stuff is going to break for a lot of people.

> Inexperienced people may use certbot and end up needing to do something. Sure. But I don't expect inexperienced people to be managing production environment servers. Do you?

Are you implying anyone using the default instructions for certbot is inexperienced?

> You claim Caddy's defaults are better - thats an opinion, but the problem is it's not a "default" it's forced fucking behaviour. You simply can't tell it to operate sanely like you can with certbot + haproxy.

If you think you can do better with your own approach, by all means do that. Most people are not PKI or ACME experts and won't have the necessary knowledge to make all the right calls, so they're better off with a "managed" solution like Caddy.

To be clear: This is an edge case and I'm in no way saying the certbot team is incompetent because they did not foresee this scenario and handle it as part of the default setup. It's just silly to go on about how terrible Caddy is when they happen to have a mitigation in place.

> Very, very few ACME clients have a renewal mechanism that supports automatic fallback to a different challenge type. I know a couple and the only one I can think of is certbot, and even there that's only true if you use the standalone or manual plugin (the others don't support "--preferred-challenges").

And that's why I use and recommend certbot in standalone mode.

> You're changing goal posts.

No, I'm trying to explain my overall point.

> Part of knowing what you're doing as a developer is knowing that you either have sane defaults or stuff is going to break for a lot of people.

Right, but in this case Caddy doesn't have "defaults" that experienced dev's can change, it has "this is just how I fucking work".

> Are you implying anyone using the default instructions for certbot is inexperienced?

I'm suggesting that anyone whose sole research into how to use ACME generated certificates in production is "what's the quick start for certbot say?" is inexperienced.

> so they're better off with a "managed" solution like Caddy.

Managed, except that they have to keep intervening or updating to a new build to restore functionality because it manages things like a drunk on a snow mobile.

> It's just silly to go on about how terrible Caddy is when they happen to have a mitigation in place.

Caddy requires a 'mitigation'. Certbot can and is deployed with a configuration that doesn't require any mitigation. It's just silly to keep claiming Caddy's approach is "the best" when you require a mitigation.

Cool. Now's a good time to remind everyone that Caddy became popular as a fully open source web server and now charges for commercial use of its binary. https://caddyserver.com/products/licenses However, there's little technically in place to prevent people from using the free version for commercial products. The result of this is that there's probably a bunch of people who downloaded Caddy when it was free for commercial use, who probably just upgraded it, and are now violating the license. A sticky situation. Better to just use nginx or apache (actually apache compares favorably to nginx when it's used properly) until another thing like caddy comes along that's free.

Ironically it's quite a similar tax to $10 SSL certificates, only it's a higher upfront cost, of $25. If people were willing to pay $25 for every HTTP server, HTTPS everywhere could have started growing quickly without LetsEncrypt.

Or perhaps now is not a particularly good time to introduce some random personal vendetta-point into a discussion of a wholly unrelated subject.

I don't get this. Is caddy so hard to build from source that you'd pay 10$ for a binary? The source code is ASL licensed so allows commercial use.

It's probably not very hard, however for an organisation or a team with several members, paying $X can be worth it - they don't have to spend effort on building and maintaining it (allows them to focus on other tasks), just deploy the binaries and contact Caddy when they need some support.

I feel like some misinformation is spread here since what you are really only paying for is building from source. The whole project is open source and you can even use commercially the source. You just have to build it yourself.

Does it not make sense that these guys would like to earn some money from this effort? That way they are also more incentivised to continue with the effort.

What kind of company finds it too costly to build Caddy themselves AND can't affort paying the license?

> Ironically it's quite a similar tax to $10 SSL certificates, only it's a higher upfront cost, of $25

Except that was $10 a year and you could use the cert on as many servers as you want. This is $25 a month per instance.

Though not relevant to the bulk of your comment, could you elaborate or provide some relevant links on why apache compares favorably to nginx when used properly?

Off the top of my head: use event MPM, disable any modules you don't need, disable .htaccess config, etc.

Apache, like nginx, is powerful and well-maintained. It has a longer history than nginx, though, and has to support some features that probably wouldn't have been implemented if the project was created more recently. One such feature is .htaccess, which makes it so an app's directory, belonging to the app's user, can configure the web server. This is a potential attack vector if the app's directory is writable (not an issue for configurations in /etc which are only writable by root). This feature can be turned off by setting AllowOverride None in /etc/apache2 (/etc/httpd on CentOS). There are other defaults that are better in nginx than apache as well. Here's a post that has the AllowOverride None suggestion and two others: https://www.jeffgeerling.com/blog/3-small-tweaks-make-apache...

nice one caddymeister