Comment by loup-vaillant | Hacker News Reader

Comment by loup-vaillant | Hacker News Reader

Parent Full thread

As the author of a non-negligible amount of peer reviewed cryptographic code, I happen to know a bit about how much safe languages help you. And the answer is, "less than you'd think".

Modern cryptographic primitives benefit very little from safe languages like Rust, compared to using C and running the test suite under Valgrind and Clang sanitisers (all 3 of them). Undefined behaviour related errors are stupidly easy to test: no heap allocations, no input dependent branches or indices, pathologically straight-line code, all that makes it fairly easy to implement an extremely effective test suite that is almost impossible to circumvent… except for logic bugs. In fact, the only significant bug I've ever shipped to production was a logic bug Rust or Go wouldn't have saved me from.

The real problems start when you start tackling encodings & textual formats. Those are definitely gnarlier, and I personally wouldn't trust myself with an ASN-1 decoder in C. This can be avoided by using simple binary formats instead, but since Age has decided to be a little more advanced using a safe language is definitely a plus.

As a text format anti-fan, I recently took a stab at defining a binary file format that's semantically equivalent to Age (more or less) [1]. It's currently just a draft/prototype/PoC, although I'd be interested to get some more eyes on it. Conceptually, it should be possible to implement compatibility with Age's plugin ecosystem, although I haven't done that yet (and I haven't thought about potential cryptographic implications of doing so, either).

I've written a bit more about my rationale and why I'd prefer a simple binary format here [2] - notably, I found two text parsing bugs (of low/moderate severity) in existing Age implementations.

[1] https://github.com/DavidBuchanan314/bagel

[2] https://github.com/FiloSottile/age/discussions/497

Just read it, I like it. I have a couple simplification opportunities:

Instead of having variable length types you could just encode them in 2 bytes magic numbers (or even 1). That should leave you plenty of different types to work with, make the encoding smaller, and is one less variable width field to deal with.

The second one I'm not sure about: starting the header with the length of the entire header, instead of marking the end with a special entry. The hope here is that recipients would know right of the bat how long the header actually is, and read it right away. They'd also know where to begin the actual decryption instead of scanning for the last special stanza.

The reasons I'm not sure about this last one because are (i) I don't know if this will actually simplify your code, and (ii) the simplest implementations would effectively limit total header size.

I did consider magic numbers for the types, however, the current version of Age is extensible without active coordination - that is, if you're designing a custom recipient type you call it "example.com/whatever", and you can be pretty sure it's globally unique, whereas with a small magic number you might collide with someone else's extension.

I didn't use a overall header length for two reasons:

- Having length fields that potentially "overlap" introduces parsing edge-cases. For example, what happens if the final recipient "overflows" the length specified earlier? Obviously you'd add bounds checks for this, but another implementer might forget!

- An implementation should scan over all the recipient fields to check that the file is well-formed before it attempts decryption. Not including the length forces parsers to do this. I don't think there's meaningful overall performance to be gained by skipping ahead anyway (since time spent doing X25519 or scrypt will dwarf the time it takes to do the parsing).

- Reason 2b - it should be possible to parse the file in a fully streaming fashion, with bounded memory usage (no dynamic allocations), without needing any hard limits. I might do a C implementation that demonstrates this concept.

> Having length fields that potentially "overlap" introduces parsing edge-cases.

Oh, that's a very good point actually. I'll keep that in mind, thanks.

> An implementation should scan over all the recipient fields to check that the file is well-formed before it attempts decryption.

Ideally an implementation should authenticate the whole header.

This means appending an authentication tag to the header, computed with from the file key. Of the top of my head, I would derive a header key and a payload key from the file key, using either a stream cipher, a hash, HMAC, or HKDF expand. Then I'd use the header key to authenticate the header. Probably using a keyed hash or HMAC to get key committent and avoid partition attacks down the line. Or, if key commitment is handled in the stanza themselves, with a fast polynomial hash.

> I might do a C implementation

In my opinion file formats should have a C implementation whenever possible: if a language as weak and as unsafe as C can handle it without too much trouble, we know it's a simple enough format.

> Then I'd use the header key to authenticate the header. Probably using a keyed hash or HMAC to get key committent

This is already the case :)

> In my opinion file formats should have a C implementation whenever possible

Yup, I totally agree. Although I haven't written any of the code yet, I've been architecting a hypothetical implementation in my head while designing the format.

Hi! I read and appreciated your issues and discussions, sorry I didn't get to respond to them yet, but I've been thinking about it.

Although I don't disagree that parsing text is hard, I also think that parsing variable-size binary formats is hard (and there is a tall, tall pile of bugs to confirm that). Really, parsing is hard. Rather than count on one design or the other to be bug-proof, I worked on a large test suite to help implementations catch their parsing bugs. [https://c2sp.org/CCTV/age] I think it would have found one of the issues you reported if that implementation had integrated it, and I am going to add vectors for various resource exhaustion scenarios which I hope would have found the other. (I am not going to look at what it is exactly, so I will know if I made the suite comprehensive enough without being too specific about this bug.)

I also liked your observation that it would have been nice if the header was streamable. [https://github.com/C2SP/C2SP/issues/28] It went on the pile labeled "regrets / for v2 when it comes", thank you.

Reply on news.ycombinator.com