Hacker News Reader

Comment by vivzkestrel | Hacker News Reader

Not gonna lie, setting up Nginx, Certbot inside docker is the biggest PITA ever. you need certificates to start the NGINX server but you need the NGINX server to issue certificates? see the problem? It is made infinitely worse by a tonne of online solutions and blog posts none of which I could ever get to work. I would really appreciate if someone has documented this extensively for docker compose. I dont want to use libraries like nginx-proxy as customizing that library is another nightmare alltogether

View on HN

I must say this is something that showcases NixOS very well.

This is all it takes to start a nginx server. Add this block and everything starts up perfectly first time, using proper systemd sandboxing, with a certificate provisioned, and with a systemd timer for autorenewing the cert. Delete the block, and it's like the server never existed, all of that gets torn down cleanly.

  services.nginx = {
    enable = true;
    virtualHosts = {
      "mydomain.com" = {
        enableACME = true;
        locations."/" = {
          extraConfig = ''; # Config goes here
        };
      };
    };
  }

I recently wanted to create a shortcut domain for our wedding website, redirecting to the SaaS wedding provider. The above made that a literal 1 minute job.

> I would really appreciate if someone has documented this extensively for docker compose

Run `certbot certonly` on the host once to get the initial certs, and choose the option to run a temporary server rather than using nginx. Then in `compose.yml` have a mapping from the host's certificates to the nginx container. That way, you don't have to touch your nginx config when setting up a new server.

You can then use a certbot container to do the renewals.

E.g.

  nginx:
    volumes:
      - /etc/letsencrypt:/etc/letsencrypt

  certbot:
    volumes:
      - /etc/letsencrypt:/etc/letsencrypt
    entrypoint: "/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'"

In your nginx.conf you have

    ssl_certificate /etc/letsencrypt/live/$DOMAIN/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/$DOMAIN/privkey.pem;

And also

    location /.well-known/ {
        alias /usr/share/nginx/html/.well-known/;
    }

For the renewals.

This is mostly why I run nginx outside of Docker, I've written about it here: https://nickjanetakis.com/blog/why-i-prefer-running-nginx-on...

I keep these things separate on the servers I configure:

    - Setting up PKI related things like DH Params and certs (no Docker)
    - My app (Docker)
    - Reverse proxy / TLS / etc. with nginx (no Docker)

This allows configuring a server in a way where all nginx configuration works over HTTPS and the PKI bits will either use a self-signed certificate or certbot with DNS validation depending on what you're doing. It gets around all forms of chicken / egg problems and reduces a lot of complexity.

Switching between self-signed, Let's Encrypt or 3rd party certs is a matter of updating 1 symlink since nginx is configured to read the destination. This makes things easy to test and adds a level of disaster recovery / reliability that helps me sleep at night.

This combo has been running strong since all of these tools were available. Before Let's Encrypt was available I did the same thing, except I used 3rd party certs.

What's the issue with nginx-proxy? We've used it for years to handle CI deploying multiple multiple Docker compose Apps to the same server [1] without issue, with a more detailed writeup at [2].

This served us well for many years before migrating to use Kamal [3] for its improved remote management features.

[1] https://docs.servicestack.net/ssh-docker-compose-deploment

[2] https://servicestack.net/posts/kubernetes_not_required

[3] https://docs.servicestack.net/kamal-deploy

the issue with nginx proxy is that i am not in control of the nginx script https://github.com/nginx-proxy/nginx-proxy/discussions/2523

i can write a simple rate limit block easily in raw nginx config but look at this mess when using nginx-proxy https://github.com/nginx-proxy/nginx-proxy/discussions/2524

I personally just terminate TLS at nginx, run nginx directly on the metal, and all the services are containerized behind it. I suspect if I had nginx then proxying to remote nodes I'd probably just use an internal PKI for that.

Usually the solution is to either not add ssl until you have the certs, or use selfsigned/snakeoil placeholder certs to get nginx started.

Personally I use dns everywhere. I have a central server running dehydrated and dns challenges every night which then rsyncs to all the servers (I'm going to replace it with vault). I kind of like having one place to check for certs

> you need certificates to start the NGINX server but you need the NGINX server to issue certificates?

I just pre-populate with a self-signed cert to start, though I'd have to check how to do that in docker.

exactly! it all sounds easy unless you want to run stuff inside docker at which point there is a serious lack of documentation and resources

Okay, it seems like if you're using compose this is now doable: https://stackoverflow.com/questions/70322031/does-docker-com... So you'd make an init container that runs something like `test -f /certs/whatever.crt || openssl command to generate cert` and tell compose to run that before the real web server container.

Reply on news.ycombinator.com