Caddy 0.10 Released
caddyserver.com
caddyserver.com
With that said, I 100% moved all of my sites from Nginx to Caddy after the config file setup is around three lines per site.
Last time I looked into it, their problem was that Caddy plugins must be configured at compile-time, because runtime plugins seem to be hard to do with Golang (besides RPC). Because there's a handful of Caddy plugins, they'd either have to provide a slim package with no plugins, or ship with all or only some blessed plugins. Obviously all approaches have drawbacks … they discussed the option of shipping their own package manager for Caddy plugins, too (shudder).
Or ship the source in the package and build the target binary in the post-install. Or do what Debian does with Exim and nginx: provide several packages, each with different set of plugins compiled in. And these are just ideas from top of my head.
Features:
- Easy configuration with Caddyfile
- Automatic HTTPS via Let's Encrypt; Caddy obtains and manages all cryptographic assets for you
- HTTP/2 enabled by default (powered by Go standard library)
- Virtual hosting for hundreds of sites per server instance, including TLS SNI
- Experimental QUIC support for those that like speed TLS session ticket key rotation for more secure connections
- Brilliant extensibility so Caddy can be customized for your needs
- Runs anywhere with no external dependencies (not even libc)
I don't think this is a reasonable summary of the situation on OS X. That being that if you turn the firewall on and then turn on block of incoming connections on 10.11 and earlier, responses seem to get dropped. Characterizing that as a blanket disallowing of direct DNS requests is misleading. (I'm not defending OS X here, it's just that summaries like this turn into cargo cults.)
It's up to you whether you trust it for large products, but so far it has been fairly magical for me.
no, that's asking if an enterprise grade project is fronted with Caddy.
> People use it for "real" projects. (I do.)
Sure, but you're not Facebook, Google or Oracle, or big brand X or Z. If you can get Coca Cola, or Nike website run on Caddy, it will change that perception.
However most companies out there aren't Facebook, Google, Oracle, Coca Cola or Nike anyway.
Here's an example how a missing option (that is Go 1.8+ - which is a recent release) can lead to a DoS: https://github.com/containous/traefik/issues/1322
(Caddy, in my experience is production quality. I've used it in a few projects and haven't had any serious issues. Maybe that's just my use cases, though.)
Just because you'll be the first to use something for some purpose doesn't make the tool unfit for said purpose.
The parent post's point stands: "production-ready", like "secure", is a highly relative term.
While Caddy doesn't do that, production readiness is something that can be objectively measured. Define some metric, like "how many % users encounter stability-related issues/error conditions", throw in telemetry and there will be solid numbers. Many do that with their own software deployments, using Sentry or similar solutions, deciding on whenever their beta deployments are "production ready" or "not yet". Of course, that only works with a large enough user base, otherwise the error margins are too high for the number to be any meaningful. It is relative only in regard whenever, say, "0.01% (just a random number) of users have encountered severe issues" is "way too many" or "oh, it's just a few".
Security, on the other hand, is a different beast. Formal proofs aside - trying to define some similar metric "how many installations were hacked" or "how many security issues were found" isn't really meaningful.
But I think it would be really off-topic. =/
The point is that knowing about previous use in production is one way of knowing to what extent something is production-ready. There are other ways (e.g.: well-controlled tests).
As for your point about security, I couldn't disagree more strongly. Comparing histories of compromises and mitigations across various projects is, in practice, a very useful metric. See OpenBSD for an example.
However, it lacks in some areas, for example if you need some complex regex-based routing it could be somewhat unpleasant (while probably doable, the configs would look really messy).
For example, FLOSS nginx does DNS lookups just once on startup. On config (re)loads, to be exact.
E.g. when you have "proxy_pass http://spam;" or "uwsgi_pass spam:9000;", you'll end up with permanent 502 (until a SIGHUP) after you've started a new "spam" Docker container and then removed the old one.
There are well-known workarounds like "set $backend "http://spam"; proxy_pass $backend;", but it's nicer when you don't even have to think about this and know that TTLs are properly honored. Caddy does that.
That doesn't sound great for zero downtime deployment but
> notices DNS changes
That sounds like it could support zero downtime (new container started, DNS records updated to this new container and then stopping old container).
Is this something like DNS SRV support in Nginx Plus? [0]
[0]: https://www.nginx.com/blog/service-discovery-nginx-plus-srv-...
Unlike Apache which served me well for 14 years without failure or interruption, I've had 3 hours of downtime in 2 months with Caddy. I haven't dug into the code, and I don't know Go, but my sense is that it needs to do much better at compartmentalising and isolating failures, and providing meaningful diagnostics for things like configuration errors.
Two specific examples:-
- If caddy can't obtain SSL certs for a configured domain, it completely fails to load. As an ops guy, I don't want 5 virtual hosts down because of a problem with a sixth, non-critical domain.
- A missing "}" in my Caddyfile caused a load failure with a message "invalid email address". Took me 30 minutes to find the problem. An error saying "Missing } in Caddyfile" would have far more helpful.
The project has great promise, but from an ops perspective, you'd be pretty bonkers to prefer Caddy over the more mature alternatives.
To me, HPKP with preloading seems like a more reliable approach (and browsers shouldn't allow this to be overridden [1]).
[1] If this breaks corporate MitM attacks, great. This practice always struck me as incredibly invasive and I'd personally be concerned if an employer was doing this. I think some traffic data should be able to be logged and timestamped in case of abuse (e.g. SNI info, TCP connections, DNS lookups) but I don't see a need to intercept application data - and honestly, it seems like the vast majority of these appliances hinder security.
If you have untrustworthy employees, you have bigger problems than you can solve with by MITM attacking everyone.
I can understand this motivation but I'm not sure in practice the SSL interception really gives you more than the traffic metadata I already described. I think this is interesting though, so I've sent an FoI request to NHS England out of curiosity to see if they're doing anything like this.
A TLS proxy that does this is meddling with the application layer and is thus broken (IMO). A TLS proxy that doesn't want to be detected shouldn't mess with the application layer.
The best thing a TLS proxy can do (other than be turned off) is preserve the characteristics of the original TLS connection exactly.
Indeed, Caddy's implementation is mainly for detecting TLS proxies that do a lousy job at it.
Traefik can efficiently deal with proxying HTTP traffic but cannot serve files and is not as easy to configure if you're not using Docker or similar.
Caddy can serve files or PHP but is not as good with proxying HTTP traffic.
https://github.com/mholt/caddy/tree/master/dist/init/linux-s...
Look at nginx, where new functionality is hidden behind a paywall. I don't want to deny them [nginx devs and sales people] their well-deserved money, but it pushes me away.
Something similar to how SVG based icons are used in many Git repos to display TravisCI scan results of the number of downloads. A similar badge could be supplied via plugin to supporters that shows how much the user has donated to the Caddy project or that they're a contributing member.
This bit our app during testing.
Caddy records the cipher suite advertised by the client during the TLS handshake and then later examines the client's user agent. Using the fingerprinting techniques mentioned in the paper, Caddy then determines whether or not the advertised user-agent is compatible with the user-agent that it inferred through the client cipher suites.
TLS interception proxies establish their own TLS connection to the server. Depending on what underlying TLS library the proxy uses, it also has its own unique fingerprint. When the TLS proxy forwards the user's request, Caddy detects the mismatch and flags it as a MITM.
After you can use consul Nginx integration (it will dynamically generate Nginx config on all updates, and will restart it).
Tyk mentioned here also have Consul service integration and much more.
https://www.youtube.com/watch?v=ZyVA9tuif4s
The way he explains things makes me wish he did tutorials for programming languages.
I'm not sure I understand. How is not having these timeouts a security thread? Someone could potentially open up enough HTTP connections to starve others from having the opportunity to do so?
I do encourage setting timeouts when you are able to. It's easy, for example:
timeouts 30s
Due to the nature of the plug system, you'll have to build caddy yourself if you add customizations like that.
Give the caddy binary the ability to bind to privileged ports (e.g. 80, 443) as a non-root user:
sudo setcap 'cap_net_bind_service=+ep' /usr/local/bin/caddyRunning as root: We advise against this. You can still listen on ports < 1024 using setcap like so: sudo setcap cap_net_bind_service=+ep ./caddy
That being said, we did run into a few issues that forced us to go back to older Caddy versions, like broken websocket support or the timeout issue in 0.9.5. Also, sometimes the documentation is a bit lacking and unclear. DNS resolution seems to be flaky sometimes (we're using alpine-based containers and sometimes Caddy just won't resolve names of other containers, even though a curl inside the container can resolve the names just fine).
So if you've got a working nginx setup, I'd say stick with it. For new projects it is worth to check out Caddy. The issues we ran into occured early in our development process, they didn't just suddenly happen in production, so once you've tested everything, Caddy just works.
Go is ona average 3x slower than C according to bechmarks, and it's hard to get rid of this issue because of how the language is architected. I just don't see Caddy as a possible long term replacement for nginx.
Is that factual, or also something you feel like?
To us simple folks, you know, it looks as if Caddy has a number of use cases where it's a much neater fit than nginx. We like the extreme simplicity and the transparency of it all. Some of us also like the Go it's written in. What with the magic Go modularity, you can actually embed this server in your web-app. Or is it the other way round? Comes in handy, at any rate.
It's the other way round.
You can embed your app in the server if you want to get its features for free, though given it uses net/http and most of the features are exposed there you should probably just use that directly instead if you are doing anything significant.
Caddy isn't the universal solution, but it sounds like you're rationalizing using the more complex tool because you know it and have been using it rather than because it's necessarily superior.
Every single thing I've done, could be implemented in an opinionated away by the web server. I'd love to just say "php" and have it do the right thing.
In fact, I'd go as far as to say, it would be amazing if I could just say "proxy ____ drupal" instead of php, and it'll add the drupal rules as well.
I can set up both nginx and apache (and get paid for it), I've automated LE for new sites via ansible, etc. Nowadays if I can do with caddy, I just use that and proceed to the next task. Also remember that easy (and short) configuration means easier maintenance.
Devs also love it, they can develop locally using advanced functionality (e.g proxying an API that doesn't provide CORS).
But seriously, for all "hobby" projects I've started recently, and even some production services now, I go straight to caddy.
My config consists of a 1 line global config file:
import sites-enabled/*
example.com {
proxy / http://localhost:5000 {
header_upstream Host {host}
}
}
I've never noticed any performance issues, I think I saw a benchmark where it said caddy can handle 2000req/s, which is at least an order of magnitude or 2 faster than any upstream service I've used it in front of.
Those are application level numbers (e.g. with logic and all).
Caddy should be able to handle an order of magnitude more req/s than that easily.
It was slightly slower than nginx, but not enough to impact typical applications
I'm sure HN would love a more in-depth comment where you say how you're accomplishing things today and what you expect out of general purpose reverse proxies, etc.
https://www.techempower.com/benchmarks/#section=data-r12&hw=...
This is on Dell R720xd dual-Xeon E5 v2 + 10 GbE. Now if I chose technology that cuts this performance 200x times I am wasting resources.
I like to idea of Caddy but before we can consider using it in production I need to make sure it can keep up with the performance requirement we got.
File-serving performance — not so good.
It supports http2 out of the box and will keep improving as the go net/http library does, performance is good (certainly good enough for 99.9% of the static sites out there, I haven't specifically benchmarked against nginx but response times didn't change much). I'd recommend it as a proxy if you need one, particularly if you need one that handles your tls certs.