Hacker News Reader

Comment by lucideer | Hacker News Reader

> I don't understand the tone of aggression against ACME and their plethora of clients.

> ACME idea good, ACME implementation bad.

Maybe I'm misreading but it sounds like you're on a similar page to the author.

As they said at the top of the article:

> Many of the existing clients are also scary code, and I was not about to run any of them on my machines. They haven't earned the right to run with privileges for my private keys and/or ability to frob the web server (as root!) with their careless ways.

This might seem harsh but when I think it's a pretty fair perspective to have when running security-sensitive processes.

View on HN

No the author seems opposed to the idea specification of ACME, not just the implementation of the clients.

And a lot of the complaints ultimately boil down to not liking JWS. And I'm not really sure what she would have preferred there. ASN.1, which is even more complicated? Some bespoke format where implementations can't make use of existing libraries?

This is exactly the impression I got here.

I would have had sympathy for the disdain for certbot, but certbot wasn't called out and that isn't what the blog post is about at all.

I disagree, the author is overcomplicating and overthinking things.

She doesn't "trust" tooling that basically the entire Internet including major security-conscious organizations are using, essentially letting perfect get in the way of good.

I think if she were a less capable engineer she would just set that shit up using the easiest way possible and forget about it like everyone else, and nothing bad would happen. Download nginx proxy manager, click click click, boom I have a wilcard cert, who cares?

I mean, this is her https site, which seems to just be a blog? What type of risk is she mitigating here?

Essentially the author is so skilled that she's letting perfect get in the way of good.

I haven't thought about certificates for years because it's not worth my time. I don't really care about the tooling, it's not my problem, and it's never caused a security issue. Put your shit behind a load balancer and you don't even need to run any ACME software on your own server.

Sometimes I wonder how y’all became programmers. I learned basically everything by SRE-larping on my shitty nobody-cares-home-server for years and suddenly got paid to do it for real.

Who do you think they hire to manage those LBs for you? People who never ran any ACME software, or people who have a blog post turning over every byte of JSON in the protocol in excruciating detail?

Our backgrounds sound similar. I just don’t sweat all those details when I set things up.

I’m not advocating for the use of cloud services necessarily, not saying we all need to allow someone else to abstract away everything. And I realize that someone on an ops team has to actually set that up at a low level at some point.

What I am saying is that there’s a lot of open source software that has already invented the wheel for you. You can run it easily and be reasonably assured that it’s safe enough to be exposed to the internet.

I gave the example of nginx proxy manager. It may be basic software but for a personal blog it’ll get the job done and you can set it up almost entirely in a GUI following a simple YouTube tutorial. It’ll get you an wildcard certificate automatically, and it’ll be secure enough.

This is the same author that threw everyone into a panic about atop and turned out to not really have found anything.

Agreed and -- in particular -- I don't recall seeing any kind of "everybody get back into the pool" follow-up after the developers of atop quickly addressed the issue with an update. At least not any kind of follow-up that got the same kind of press as the initial alarm.

Im not a container guru by any means (at least not yet?) but would docker not suffice these concerns?

The issue is that the client needs to access the private key, tell web server where various temporary files are during the certificate generation (unless the client uses DNS mode) and tell the web server about a new certificate to reload.

To implement that many clients run as a root. Even if that root is in a docket container, this is needlessly elevated privileges especially given the complexity (again, needless) of many clients.

The sad part is that it is trivial to run most of the clients with an account with no privileges that can access very few files and use a unix socket to tell the web server to reload the certificate. But this is not done.

And then ideally at this point the web servers should if not implement then at least facilitate ACME protocol implementations, like, for example, redirect traffic requests from acme servers to another port with one-liner in config. But this is not the case.

Apache comes with built-in ACME support. Just enable the mod_md module: https://httpd.apache.org/docs/2.4/mod/mod_md.html

But the requirements you listed aren't actually requirements of ACME, they're lazy choices you could make but they aren't necessary. Some clients do better.

For example the client needs a Certificate Signing Request, one way to achieve that is to either have the client choose the private keys or give it access to a chosen key, but the whole point of a CSR is that you don't need the private key, the CSR can be made by another system, including manually by a human and it can even be re-used repeatedly so that you don't need new ones until you decide to replace your keys.

Yes, if we look back at my hopes when Let's Encrypt launched we can be very disappointed that although this effort was a huge success almost all the server vendors continued to ship garbage designed for a long past era where HTTPS is a niche technology they barely support.

I don't know that it's accurate, but at the beginning, it felt like using certbot was the only supported way to use ACME/LE, and it really wanted to do stuff as root and restart your webserver whenever.

Or you could run Caddy which had a built in ACME client, but then you're running an extra daemon.

apache_mod_md eventually came along which works for me, but it's also got some lazy things (it mostly just manages requesting certs, you've got to have a frequent enough reload to pick them up; I guess that's ok because I don't think public Apache ever learned to periodically check if it needs to reopen access logs when they're rotated, so you probably reload Apache from time to time anyway)

Before that was workable, I did need some certs and used acme.sh by hand, and it was nicer than trusting a big thing running in a cron and restarting things, but it was also inconvenient becsause I had to remember to go do it.

> I don't know that it's accurate, but at the beginning, it felt like using certbot was the only supported way to use ACME/LE, and it really wanted to do stuff as root and restart your webserver whenever.

It's fair to say that on day one the only launch client was Certbot, although on that day it wasn't called "Certbot" yet so if that's the name you remember it wasn't the only one. Given that it's not guaranteed this will be a success (like the American Revolution, or the Harry Potter books it seems obvious in hindsight but that's too late) it's understandable that they didn't spend lots of money developing a variety of clients and libraries you might want.

It's cheap. If the client was done today, it would be based on AI.

Yes, it does.

I run acme in a non privileged jail whose file system I can access from outside the jail.

So acme sees and accesses nothing and I can pluck results out with Unix primitives from the outside.

Yes, I use dns mode. Yes, my dns server is also a (different) jail.

My reading of the article suggested to me that the author took exception to the code that touched the keying material. Docker is immaterial to that problem. I won’t deign to speak for Rachel By The Bay (mother didn’t raise a fool, after all), but I expect Docker would be met with a similar regard.

Which I do understand. Although I use Docker, I mainly use it personally for things I don’t want to spend much time on. I don’t really like it over other alternatives, but it makes standing up a lab service stupidly easy.

I use docker for the same reasons as the author's reservations - I combine a docker exec with some of my own loose automation around moving & chmod-ing files & directories to obviate the need for the acme client to have unfettered root access to my system.

Whether it's a local binary or a dockerised one, that access still needs to be marshalled either way & it can get complex facilitating that with a docker container. I haven't found it too bad but I'd really rather not need docker for on-demand automations.

I give plenty* of services root access to my system, most of which I haven't written myself & I certainly haven't audited their code line-by-line, but I agree with the author that you do get a sense from experience of the overall hygiene of a project & an ACME client has yet to give me good vibes.

* within reason

Copilot suggests:

    docker run --rm \
      -v /srv/mywebsite/certs:/acme.sh/certs \
      -v /srv/mywebsite/public/.well-known/acme- 
      challenge:/acme-challenge \
      neilpang/acme.sh --issue \
      --webroot /acme-challenge \
      -d yourdomain.com \
      --cert-file /acme.sh/certs/cert.pem \
      --key-file /acme.sh/certs/key.pem \
      --fullchain-file /acme.sh/certs/fullchain.pem

I don't know why it's suggesting `neilpang` though, as he no longer has a fork.

Yeah I'm not running anything llms spit at me in a security-sensitive context.

That example is not so bad - you've already pointed out the main obvious supply-chain attack vector in referencing a random ephemeral fork, but otherwise it's certonly (presumably neil's default) so it's the simplest case. Many clients have more... intrusive defaults that prioritise first-run cert onboarding which is opening more surface area for write error.

Reply on news.ycombinator.com