I still love nginx, but it's a damn shame they gate some really useful features behind their nginx plus license, namely service discovery via DNS SRV records, cache purge requests and request queueing for upstreams.
My employer (Wikimedia, which is a non-profit) has been running all our server infra on pure open source for many years, for very principled reasons. We even publish all of our internal infrastructure config in public git repos as well. We're an open-source-only shop that's doing very non-hobby work. We have thousands of servers, we run our own CDN on multiple continents, we have somewhere around a billion unique users per month, and our average global rate of inbound HTTP requests is around 135K/second.
However, it keeps getting harder to stick to our open source principles every year as more projects shift towards open-core models with this kind of "Anyone who needs feature X is surely a big corporation making billions who doesn't mind running closed source software" type of thinking. It's not just bad for us, it's bad for the whole larger open source and Internet ecosystems if all the important bits for running at scale are locked up and hidden away in closed-source software.
Avoiding recurring payments you can't easily stop makes reasonable financial sense to me. Particularly for a business that doesn't sell anything.
Many people assume that developing it in-house is a one-time cost. However, in my experience, code is not an asset but a liability for which you will have to pay interest in the form of maintenance. More specifically, in this case there will most likely need to be adjustments made to the in-house extension as the oss nginx version it was built on top is being updated over the years.
My point here is that you should probably treat in-house code as a recurring cost, because in practice it usually is.
The question then becomes which recurring cost is higher: The license or the in-house maintenance?
Accepting these "sprinkles on top" licenses is highly foolish. I do hope that the true free and open source licensed software prevails in the server space. As many as possible must continue to hold out and push forward with new better and better scalable fully open source software.
Curious how deep does it go? Do you only use firmware in server which has open source code. Do you require all drivers to be open source?
> It should be possible to run large-scale important things on pure open source.
It is definitely possible. With enough developer time, you could create copy of any software. Every paid or open core software runs on the philosophy that it is cheaper to pay them rather than to build them.
Which brings to the core issue, developer time is not free or cheap. The fact is that software quality and reliability is dependent on the money that the software has means paid software quality in general should be higher. While wikimedia could get enough money through donation, no one donates to open source projects that could be counted in full time developer salary, likely not even wikipedia(correct me if wrong).
It has been built by extending Nginx with an OpenResty-based DNS resolver instead. You could even extract it away from Kong itself and use it standalone.
It took me almost exactly 3 minutes start to finish with letsencypt. Where 'start' was "I should stop putting that off" and typing "letsencrypt.com" into a browser, and 'finish' was nginx serving up https:// on all my domains. I'm genuinely curious what nginx could possibly do to improve the situation there.
You don't have to mess with having a .well-known always accessible on each domain, for example. Caddy does it for you and only when it's needed.
Nginx has a profit from big business that would require coordination to avoid multiple certificates. For comparison, it is worth noting that to ensure such coordination, Traefik requires an Enterprise plan and a separate agent. Such a business also often has mechanisms for storing certificates, so they do not necessarily want to integrate it into Nginx.
Still irritated that they ship it with an unprotected admin endpoint enabled by default on localhost:2019 that eats JSON and allows reconfiguration of the webserver like adding new sites that enable further attacks.
Put "{ admin off }" as a separate block in the root Caddyfile to disable it.
That said, see https://github.com/caddyserver/caddy/issues/5317, we are considering changing the default, but that would be a breaking change although it would likely be a transparent change for most users. The default that makes the most sense depends on the platform and installation method, which is why it's complicated.
Glad to hear you're considering changing the default!
It would be enough for me if Caddy generated a password (that's hard for attackers to predict) on first launch, set that in a config file it has write access to (autosave.json for example), and then required Basic auth using this password unless the configuration specified otherwise. My problem is that this endpoint is entirely unauthenticated.
We have yet to see this, but if we do see a practical demonstration, we're happy to reconsider.
The problem here is that a priviledged process (and yes, a process that has access to Port 80 and 443 is priviledged even if it doesn't run as root) gives unauthenticated control to less priviledged processes.
With good security design you don't lock things down as needed but instead start fully locked down and open up only the accesss you really need.
This. @mholt what we're arguing for is just to have secure defaults.
What's the pain?
https://doc.traefik.io/traefik/providers/docker/
(but preferring Caddy so far)
I should revisit Caddy and see how it gets on.
Also, there's massive advantages to having TLS issuance built into the webserver, such as proper OCSP stapling, having an active process to trigger renewals as soon as a revocation happens (hearing about it via OCSP), solving the ACME TLS-ALPN challenge without extra steps, and unique features like On-Demand TLS that many SaaS companies are relying on to provide a custom domains feature to their customers. None of those things are possible unless it's tightly integrated in the webserver.
Or docker: https://eff-certbot.readthedocs.io/en/stable/install.html#ru...
There are quite a few other options besides certbot, I was just suggesting it as a "install + one liner" for adding auto renewing ssl certs to a server.
Well, for what it's worth, certbot is still pretty good: https://certbot.eff.org/
That said, with servers like Caddy supporting automatic TLS and even Apache httpd having built in support now with mod_md, Nginx feels more like an outlier for not supporting ACME out of the box: https://httpd.apache.org/docs/2.4/mod/mod_md.html
For my personal sites I actually use Apache because it's pretty good and has lots of modules (including OpenID Connect support for Keycloak etc.), but Nginx has always been really easy to install and setup, especially for serving static files (e.g. built front end apps) or being a basic reverse proxy.
Here's a bit more about how I use Apache: https://blog.kronis.dev/tutorials/how-and-why-to-use-apache-...
And a little bit about some of the occasional warts of Nginx: https://blog.kronis.dev/everything%20is%20broken/nginx-confi... (though otherwise it's fine)
That said using HTTP-01 instead of DNS-01 across multiple nodes with the same hostname is needlessly complex in most cases and DNS-01 doesn't always have the best support in web servers (Apache in particular is just like: "Yeah, we can run whatever script you want, but the contents and integrating with your DNS server is up to you").
Pay for the features you want. You will be better off for it. Expecting every company to follow the RedHat model is unsustainable.
Vs just actually selling the software and having it work flawlessly with minimal work.
Always
If it's actually gated, not enough people care enough to get together and make and maintain a better one themselves.
Literally, at work (a startup), we use Traefik (which is horrible imho) instead of nginx (which I've used literally everywhere else) because the devs who originally started this project had never used it on a personal project.
So the issue isn't paying for it, it is about making it useful enough for personal projects that people use it for personal projects that it then gets used on professional projects. Right now, it is barely useful for personal projects unless you know what you're doing.
Do I personally need DNS SRV support? No, I have a templated Nomad config that will re-render and reload the nginx config if the consul upstreams change. Setting this up though is definitely a bigger hurdle than just specifying a Consul service as an upstream.
It seems they're more interested in high-touch sales (which is unlikely to happen on a dev team), vs. organic growth.
A vendor that releases some free software and some proprietary software is a proprietary software vendor. (This describes, for example, both Microsoft and nginx.)
The cherry on top is the OpenResty “community”. You know the one that call you an idiot for not pretending Lua is orgasmically good; the one that declares you “just” implement thousands of lines of code because there’s no possible way a useful library could exist; the one with the ecosystem full of awful, worse than js leftpad garbage that hasn’t been updated in three years.
Give me a break. F nginx and openresty.
Caddy has sane defaults for many settings and I only need to add a couple of response headers, drop the domain name, and where to proxy the requests to. It takes care of maintaining things like the list of TLSv1.2 ciphersuites.
Adding more domains takes two lines of config thanks to parameterized includes — another important thing nginx misses that has to be implemented manually via scripts using something like envsubst (or again, full-on automation which is problematic if many of your servers are just a little bit different from each other — for good reasons).
Apache httpd has it by the way: https://httpd.apache.org/docs/current/mod/mod_macro.html
I really wish companies would come up with SMB pricing to help us side project hackers out so we could grow into paying for the bigger plans. Also, I understand why they don't. SMB support is a huge PITA.
We do pay them, for a bunch of boxes. But I can't afford to run their plus version in front of every microservice. Or even in front of our entry point server. That's 5 times as many machines, plus staged versions of the app, plus other services running on the same boxes.
If you're running anything that is not multithreaded, you're running a reverse proxy in front of it. And even then you can still improve server availability by running a reverse proxy.
While one can talk about corporate contributions and basically ownership of some open source projects (the F5/Nginx case included), I also think that the funding is a big challenge for many projects out there.
Now, this is from 2019, but I found the article "Software below the poverty line" to be a useful if a bit grim look at the way things are: https://staltz.com/software-below-the-poverty-line.html