Hacker News Reader

Not at all: adding new packages or developers is automatically signed (by the snapshot bot). Administrators may later sign the keys and delegations, individually or in batch, after verifying the developer identity out of band (the specification doesn't yet include the mechanisms for this verification, but with a git repo based on GitHub, the GitHub login gives a good start).

Thank you for the clarification. Can I install these "not yet trusted" packages? Presumably I can, I just get a big warning. And how do you deal with squatters - can I (as the "real developer") submit a second claim on an unverified package name (that an impostor has claimed)?

Edit: And does this mean that the snapshot bot automatically merges (well-formed) pull requests for new packages?

The pull-requests for new packages are manually merged (as it the case now), automatically signed.

Names uniquely identify one package. A repository maintainer can replace the owner of a package, and clear the existing package contents. This shouldn't be done for a different package with the same name if any users still have the package installed, though, as they will consider that an up/down-grade.

Hope that clarifies a little!

That does help me a lot - thanks! The manual "pre-approval process" via accepting the PR is what I was missing, and that does seem to fix most of the shortcomings of the underlying system (assuming a reliable human!) So you have a human that looks at incoming requests to prevent abuse, and to arbitrate problems. And then the snapshot server signs that result to allow distribution over HTTP, and then periodically it gets signed by the quorum of "gods" to indicate that these packages have the full seal of approval.

I think it is pretty specific to OCaml's current workflow and volume, but signing is better than not signing!

It's indeed specific to our workflow -- allowing additions without moderation would cause all sort of problems we don't address; but not to OCaml, e.g. Homebrew uses a very similar, GitHub based workflow (which we actually drew lots of inspiration from) !

What is nice is that the moderation, if given the proper reports, e.g. Travis with lint and security checks, is very fast in the simple cases, and readily opens discussion in the others -- that's where GitHub really shines.

I mentionned GitHub, but note that we intend to also provide the tooling to host your own signed repository, indenpendently of it.

You should briefly think how opam does updates currently: a repository on GitHub where developers submit their pull requests. This won't change too much.

Guarantees are: if a new package from a new developer is submitted, basically none. The signing of delegation and developer keys (by repository maintainers) enables a web-of-trust style system: once package `foo` has been claimed by the well-known (and signed) developer `alice`, there's no way that `eve` will successfully update `foo` (without getting `alice` key, and also then need to go via GitHub and repository maintainers).

Unrelated to my reservations about this system, I think the idea of computation combined with a git datastore is very interesting and possibly more generally applicable. I like the idea that the snapshot bot enforces additional guarantees on the evolution of the repository, like the ones you're describing here.

Off-topic but have you come across Irmin? It seems like something you might be interested in.

https://mirage.io/blog/introducing-irmin

That's the sort of thing I'm very interested in - thanks for the link. I'm particularly interested in doing this actually over git, so the mapping to git is great (as you can probably infer, I think there are big advantages to leveraging existing "devil you know" systems!)

I cannot follow your claim "stronger procedures and guarantees than what we are getting here". CAs might be useful for a single resource, but we have here a bag of packages and want to distribute the whole repository (with end-to-end signatures). Also, please read the TUF paper and specification which elaborates on the motivation.

And serving sth over http/s and the repository out-of-band opens possible attacks - that's why we decided on a single connection. Git gives us already some linearity/monotonicity conditions.

The fundamental problem here is signing keys as authoritative over subsets of a namespace (I believe). CAs are pretty good at that, so we should compare TUF to the CA system (with pinning). In practice, that means that you have to get a certificate for your package, which also requires a DNS name and SSL hosting. You could also use the PKI infrastructure, but only to sign the packages, and not require TLS. If you wanted to eliminate the cost requirements, you could offer subdomains of opam.ocaml.org, (and make sure that your CA would sign sub-sub-domains). But then you're taking on the burden of allocating those subdomains.

My fear is that even allocating subdomains of opam.ocaml.org is a tough problem in the real world. And you're not only doing (the equivalent of) that, you're also running an alternative CA, with alternative signing software. I wish you success, but this seems ambitious.

In contrast, we know the CA/https system works and we know its failings (in particular the evil CA attacks, mostly fixed by pinning). That system is highly distributed and needs no custom software. It also needs little extra work, which is always a good thing.

I've looked at the TUF papers. Can you elaborate on the attack you're describing? They don't seem to compare TUF to alternatives (that I could find). (Edit: and to be clear, my straw-man here is serving the initial git revision information over HTTPS, fetching the hashed data over HTTP & mirrors, and then verifying the hash)

It is not clear to me what you are reading in the proposal. There is no need for DNS and SSL hosting. Packages are put as tarball onto the Internet. Then in the opam-repository the metadata, including a signature over the tarball. The tarball is fetched (over any insecure connection) and its signature is verified. The naming and trust delegation happens within opam and the opam repository.

How is an update supposed to work if the connection to the timestamp is prevented/intercepted? How if the connection to the repository is prevented/intercepted? Using a single connection leads to less potential failure modes.

I understand that; I am trying to explain one way that a CA-based system might work in response to your statement that the CA system was not applicable. I believe I understand the basics of the TUF system, though the mechanics of your update mechanism are still not entirely clear to me. I simply haven't seen a comparison to the (more naive?) alternatives like the one I am straw-man proposing.

I think that TUF and an https based system would both warn if an adversary or adverse conditions was blocking connectivity to the "root file". I don't think either can do anything more than warn. TUF makes it easier to mirror the root file (because it doesn't require HTTPS), so TUF has the advantage there. But I think an attacker could probably as easily (more easily?) block or intercept every TUF mirror (over HTTP) as they could block a small number of HTTPS update servers, so I'm not sure this is a huge advantage in real life.

I believe the same problems apply to delivery of the root file as to subsequent files; if you are concerned about malicious attackers interfering with file delivery it seems you should prefer HTTPS. Both TUF and HTTPS-for-root are basically agnostic to the delivery mechanism for subsequent files, I believe, so I don't see this as an advantage for either: for both an attacker can trivially block individual files over HTTP; if you use HTTPS as the transport it is still trivial to block connectivity altogether, and neither approach offers a solution to this no-connectivity attack.

opam could show a warning if it didn't have an updated signed timestamp for a certain amount of time (1 day, 7 days). We had a similar warning for ClamAV when no anti-virus signature database updates were seen for a certain amount of time, usually indicating DNS problems or bad mirror selection.