Show HN: Monocypher: a small, easy to use crypto library
loup-vaillant.fr
loup-vaillant.fr
So perhaps the marketing is off: I don't view this as a "small, secure, auditable, easy to use crypto library"; I view this as a practical and ongoing study in investigating how to make small, secure, auditable, easy-to-use crypto libraries. To be fair, the page mentions that it's "not ready for production" and needing external review a quarter of the way down, but if the point is to prove that a sufficiently dedicated individual can learn the material, endure some criticism, and produce quality software following the example of well-acclaimed experts, then own this point, bare your disclaimers in the beginning, and make it more prominent.
And once you phrase it that way -- which is arguably consistent with the author's previous posts (and HN comments) on the topic, it's suddenly a lot more palatable, and regardless of its ultimate outcome, I can only applaud.
[1] I use an interpretation of "don't roll your own crypto" that's slightly less circular than the author's. In my opinion, it's shorthand for 'defer to the opinion of experts, use ready-made constructs when possible, and if not, then exercise caution when hooking crypto primitives together in unproven ways'. Originally posted here: [3]
[2] http://loup-vaillant.fr/articles/rolling-your-own-crypto
On the other hand, I don't see how I can ensure my library is bug-free without external help.
"Probably bug free" is not good enough. I need over 99.9% certainty, and I'm currently pretty far from that.
That's mostly to say: I agree with your assessment that you (and me and everyone) need help to review code and look for bugs.
Do you mean that the crypto algorithms work as advertised? e.g. would DES after public knowledge of differential cryptanalysis be okay?
Do you mean that you have implemented algorithms exactly as specified?
Do you mean that you have implemented algorithms exactly as specified that are also constant time so that they resist timing attacks?
Do you mean that you have implemented algorithms exactly as specified that are not vulnerable to cache eviction attacks on shared hardware?
Do you mean that you have implemented algorithms exactly as specified that resist bit-flip attacks or SDR attacks, or audio attacks, or other Van Eck style listening?
I'm actually not trying to be down on your work. It's hard work, and the world needs more people who actually do it. I'm just saying it is VERY hard to feel confident, and as someone who has a passing interest in this kind of security, I worry when I read statements like you made.
That changes when the software does things previously impossible or hard. NaCl and libsodium were interesting not because you couldn't do strong and safe AEAD before (OCB is from 2012, AES-CBC + HMAC-SHA256 EtM have been available for almost two decades) -- it's because you finally had a realistic chance of getting random programmers to do AEAD in some code and have it be OK.
Noise, hence is interesting because it gives you a transport with interesting new properties, including forward and backward security that you don't get out of the box with NaCl/libsodium. E.g. if you build a naive session protocol with libsodium, odds are you're vulnerable to KCI attacks. That's not a bug in libsodium because it never promised to do that, but it would be a bug in Noise.
Similarly: STROBE! New, exciting! Low-footprint crypto! Strong tools! Also not recommending anyone suddenly use STROBE for everything (especially not in software, but we can talk if you have embedded hardware problems).
First: it needs to be made clearer why anyone would use this rather than Nacl/Sodium.
Second: this statement does not inspire confidence:
"The current code is fairly rigorously tested. Every primitive match their respective test vectors, and most constructions are tested for consistency. The entire test suite is Valgrind clean."
If your primitives don't match their test vectors, your tool doesn't work at all, so pretty much every crypto tool, including MiLiTARY-GRADE-CRYPTO v20 GOLD on the Android store or whatever, achieves this level of rigor.
Here's a good Reddit comment on the last crypto thing the author wrote, a few weeks ago:
https://www.reddit.com/r/programming/comments/5iv1ti/rolling...
The advantages are marginal. Negligibly better primitives (Chacha over Salsa, Blake2b over SHA-512), ease of deployment (only one .c and one .h file), and cleaner interface (foolproof functions return void). Oh, and it has an Argon2i implementation that doesn't allocate.
> Second: this statement does not inspire confidence:
Actually, that's kind of intended. When I say "fairly" rigorously tested, I mean users should stay the hell away, at least for now. Right now I need reviews, specific counsel, criticism, possibly even audits. I've went as far as I could on my own.
> If your primitives don't match their test vectors, your tool doesn't work at all
Well of course, but you wouldn't believe how strong the temptation to punt on those test vectors can be. On the other hand, test vectors are surprisingly complete. Almost all of my primitives went from "utterly broken" to "probably perfect" with a few character changes. Heck, those test vectors even let me find a bug in the reference implementation of Argon2i.
---
I'm close to the point where I'd trust this library with my own data. There's a good chance this library is bug free. There's just no way I'd trust it with your data just yet.
Tests on test vectors is the first thing I write, before I even start filling in the implementation of a primitive. I don't do this out of principle, but because I've never managed to get an implementation of a primitive right without it.
Parent is right, test vectors are almost part of the declaration - and don't really count as actual testing.
Side-note: Props for your humility towards criticism, both here and on reddit.
> only one .c and one .h file
If your library gains popularity, I predict this will be one of the largest contributing factors. In particular for embedded projects, the ability to just plop in a dependency free file is huge ("programmers are like water" and "path of least resistance" etc..)
My journey looked a bit like this. First implement all the thing! Cool it compiles! I'm tired… Okay, let's do the test vectors, because crypto-critical-yada-yada. Aw crap. It was all broken. Okay, let's dig it up…
By the way, one worry I have right now is that I may not have enough test vectors. Do they hit all the code paths? Do they sufficiently test for overflow? That's why I display the number of test vectors involve in each tests when running the test suite.
I don't think so. Check out the test suite in my tarball, there are files named "vectors", with sets of inputs and expected outputs.
In addition, I have written consistency checks for all constructions (encryption-decryption roundtrip yields the original plaintext, legitimate messages are accepted, forgeries are rejected…)
I feel I should do more, but I'm not sure what exactly.
I'm not saying you shouldn't write tests for misuse cases! You should do that.
Fuzzing tests can also uncover weird bugs and edge cases that can lead to vulnerabilities.
Regarding the Reddit thread, I actually enjoyed a response much more than the comment you linked:
https://www.reddit.com/r/programming/comments/5iv1ti/rolling...
I would agree if the author presented a very clearly flawed and poorly thought out implementation, but this is clearly not the case here. There is a quality threshold after which spouting out "No, use NaCl!" becomes a waste of time.
> by their own admission is not a significant improvement over TweetNaCl?
Not everything in the real world has to be a significant improvement.
> [...] if something doesn't do a good job arguing why you need it
Not everyone is good at marketing his/her creations. It's pretty tough to do well in my opinion, and there's always "that guy" who's never satisfied by any explanation.
> [...] _solid engineering advice_
I would agree with you in most other scenarios, but the author of this library mentioned NaCl in the first paragraph. The author also took the time to list the motivation behind this work, as well as the reasons why this could provide potential improvements over prior work.
So I don't see how "NO, use NaCl!" is 'solid' advice in this case.
It is not true that any attempt at new crypto software will just be shut down; for example, I think Noise is awesome. I think STROBE is pretty cool too, but I'm not telling people to implement STROBE. I'm telling people to use libsodium. Telling people to use the probably-already-packaged, carefully reviewed, random-nonce supporting package they had available yesterday _is_ solid engineering advance, and I stand by that claim.
I still got a very "stop what you're doing and move on" vibe from the parent comment as well as the response on the Reddit thread, and I feel that it wasn't fair in this case.
I apologize if I misunderstood.
> First: it needs to be made clearer why anyone would use this rather than Nacl/Sodium.
... and a comment about how a few test vectors is not a serious cryptographic audit.
So it seems that the parent agrees with you, and nobody told anyone to stop doing what we're doing. The thing we (well, I; I don't want to put words in tqbf's mouth) did say repeatedly, is "the answer is still libsodium". It seems like that, too, is something you agree with.
Wait a minute, what's that? Does it just mean nonces big enough to be random, or something more I should consider for inclusion in Monocypher?
That one is a tough call for me. If I brag too much, I can be dangerous. Too little, and nobody uses my stuff. I'm not sure about the significance of the improvements, but they are undeniable: except for curve25519, everything is more readable, faster, and a bit more useable than TweetNaCl (curve25519 is merely more usable).
There is also one significant contribution I did not brag too much about: my Argon2i implementation is much simpler than the reference implementation. I'm considering adding support for multiple lanes (maybe with threads, maybe not), and Argon2d. The result should be RFC worthy.
Was re-implementing Argon2i from scratch good engineering practice? I don't know. What I do know is that I ended up uncovering a long standing bug in the reference implementation; one that complicates implementations and hurt security a little bit.
I think Monocypher was worth the effort, even if just for Argon2.
Re: usability of other core primitives: I'll bite. Places where you don't get to solve a hard coordination problem to agree on a nonce matters. I know you document that you can't use a random nonce (and that's accurate), but I think that is an important safety regression.
Looking back at my manual, I understand it was easy to miss. But you can. You just have to use XChacha20, which gives you a 192-bit nonce, where the chance of collision even after 2⁶⁴ messages is negligible. The high level constructions all use XChacha20, so random nonces are possible.
I really should re-arrange the order of presentation.
The slightly better usability mainly referred to the fact that functions that cannot fail return void. It's small, but the user is not left wondering whether the return value should be checked, or why the manual doesn't say anything about it. I tried to make the ordering of arguments consistent across the whole library. Easier interfaces have shorter names (no "easy" suffix).
Then there are opinions. Primitive functions are named after the primitive, because I don't believe in hiding those to the user. Fixed sizes are hard coded, because I don't expect them to change without changing the primitive itself. Constants like CRYPTO_ENCRYPTION_KEY_SIZE obscure the point somewhat.
In the present case however that will be harder. We're talking over 1K lines of extremely critical code, and that's much harder to review than a 300 lines blog post.
But if you insist, I can request that HN mods delete my comment, as I can't edit/delete it anymore.
I've previously criticized a blog by the same author called Rolling Your Own Crypto, which had serious flaws.
AFAICT, this is ostensibly cryptographically fine (modulo some safeguards that probably only matter if you're Doing It Wrong already anyway), even though the final recommendation is still "avoid". There are many things I disagree with (don't put the documentation for the internal hash for ChaCha20 before the authenticated encryption API!), and using TweetNaCl as a base is less interesting than using libsodium which is both fast and has a lot more eyeballs on it.
One .c file, one .h file. No fancy compiler option. It doesn't get any easier. I agree with most of the rest, including the "avoid" recommendation, since Monocypher had only one pair of eyeballs so far.
> There are many things I disagree with
Please, I need more! (The documentation order was out of laziness, I'll change it.)
> using TweetNaCl as a base is less interesting than using libsodium which is both fast and has a lot more eyeballs on it.
TweetNaCl is smaller. Much smaller. It doesn't need nearly as many eyeballs as libsodium to reach the same level of confidence. And I'm not sure libsodium had as much audits and static analyses as TweetNaCl, if only because its size may render some analyses impractical.
I tried to use ref10 as a base. It's too much code.
It may be true that it doesn't need as many eyeballs; alternative implementations of the same code have the benefit that they're pretty easy to automatedly check. libsodium is not anywhere near the size where static analyses become impractical -- do you have any evidence that would support that?
The C interface is the first step. I may consider C++ and OCaml bindings in the future. Still, point taken: calling C from other languages is always a bit of a pain.
> libsodium is not anywhere near the size where static analyses become impractical -- do you have any evidence that would support that?
It's not much, but the TweetNaCl papers indicate that its size made it possible to automatically prove the absence of a couple classes of bugs with an algorithm prone to combinatorial explosion. Even being twice as big could have been too much for those.
For those use cases, the consumed artifact is a .so. Having a .so is easier than having to make it. Furthermore, having repeatedly built libsodium (I built FPM debs when libsodium wasn't yet commonly packaged), for those people building this is just as easy as libsodium if for some reason they need to build at all.
What's the point, a compilation unit is only a gcc call away from being an .so, right? Besides, I made sure I don't expose unnecessary symbols to the linker. What you're telling me amounts to saying binary distributions are easier to consume than source distributions.
I would be quite bewildered by such a claim. As well as a little upset at my disadvantage: I do not have the resources to package my library for all major OSes and distributions. This of course has nothing to do with the library itself, merely the community around it (with one author and no user, as is the case with anything new).
I've seen new programming languages being criticised for the immaturity of their tooling or the lack of community. This feels similar.
---
Or, your idea of an .so is different from mine. Does .so libraries have something the C source files don't?
This assumes two things:
1) that most users are indeed going to use this via FFI, which (to me at least) sounds like a very bold claim
2) that the target language provides easier support for precompiled shared libraries than it does for using an arbitrary .c or .h source file. This is not always (or even usually) the case, especially if the language's FFI mechanism comes from the C side rather than the target language side (see also: Perl (pre-6) and Ruby when not using third-party FFI modules) or if there are a lot of foreign functions needing an interface (in which case one will likely use some kind of generator, which usually requires the use of a C header to provide the foreign definitions).
http://loup-vaillant.fr/projects/monocypher/manual
Example from crypto_chacha20_init (grammar aside):
"Warning: don't use the same nonce and key twice. You'd expose the XOR of subsequent encrypted messages, and destroying confidentiality."
Brought to you by the person that wrote "Rolling your own crypto" (see earlier discussion linked below), with the motto "Impossible? Like that would stop me."
Some fields of knowledge in certain phases benefit from iconoclasm and Sturm & Drang, but is cryptography one of them? On the other hand, many existing implementations are so crufty that a fresh wind might help (if only by pushing more experienced cryptographers to put out better implementations).
At any rate, at least the writer followed through, though I'd not be inclined to use this in production anytime soon.
The answer seems "no" to me.
Indeed you should not. Rolling crypto on your own is too damn dangerous, and that's what I just did. On the other hand, I'm convince there's not much left to do to ensure a sufficient level of confidence, for 3 reasons:
1) The library is small. Almost as small as TweetNaCl itself.
2) With the exception of Poly1305 and curve25519, the primitives chosen here are simple and easy to test. If I didn't royally screw up somewhere (and I need external verification to make sure I didn't), those primitives are all production worthy.
3) curve25519 is is directly from TweetNaCl with very few modifications. The hard part (ensuring there are no rarely occurring limb overflow) has already been done there.
The weakest link for now is Poly1305: limb overflows are possible, and I did it from scratch. There's no way I can trust it without external review.
Some notes:
- Let's call Ed25519 with BLAKE2b something like "Ed25519-BLAKE2b" to avoid confusion.
- crypto_chacha20_random is a confusing name, I'd prefer crypto_chacha20_stream
- Everyone uses "crypto_" for namespacing their libraries nowadays! Let's use something else to avoid collusions :)
We may be able to compete with ref10 without too much code. I haven't found a way yet (very frustrating). In the mean time, you have the Donna implementation, which can be used with any hash (it's a compilation option).
- I'll consider crypto_chacha20_stream
- I'll stick with "crypto_", for 2 reason. First, you only need one crypto library. Second, a brutal search & replace will let you use another namespace in 30 seconds. Maybe I should provide a script to do that?
Why? If the goal is compact, easier to deploy - why have choice at all?
[ed: I just saw this on the homepage:
Note the presence of SHA-512 in a separate compilation unit. It is meant to test ed25519 with the test vectors I got from the RFC. In production, most users will be expected to use Blake2b —the default setting.]
I also wonder a bit on the naming - using algorithm over intent (sha512 vs "hash") - I can see why transparent naming is good - on the other hand, wouldn't the easiest approach be: use some sort of framing (monochypher_v1) and pair that with primitives that "do the right thing"?
I suppose one could use monochypher as a provider and another "interface" library as a consumer - might make for easier auditing and development.
But now you need two header/c files, and they have to stay in sync in order to be reviewed in pairs (to avoid subtle bugs in key handling and such?).
Which brings us back to: in five years, when a developer needs to update their simple utility that uses monochypher - will that developer have to consider changing cryptographic primitives, moving users from one generation of ciphers to another - or will monochypher handle that?
I need SHA-512 for tests. The only test vectors I could find were for the official ed25519, with SHA-512. The option is there anyway, I might as well provide it.
I do transparent naming because the choice of primitive has user visible effects. If I have the SHA-512 of some file, I can't just use Blake2b to check that hash. Same for encryption and authentication: changing them is transparent only for transient messages. Encrypted files won't convert themselves to the new encryption scheme, and neither will the authentication tags. Same thing for long term secret/public key pairs.
That said, the high level constructions don't mention the primitives. They're all called "crypto_lock" or something.
Long term, changes are not expected to be transparent. The only backward compatible path is to stick to the older version of Monocypher. Any transition will need to be handled explicitly. I think that's okay, because I don't think I'll need to make breaking changes often. I expect a stable version to last at least 5 years, possibly 15. Then the next version will break everything, and users will just have to cope, most probably by supporting both v1.0 and v2.0 for a time. Hopefully that won't be too hard, given the small size of the library.
Documentation took me relatively little time. Probably a week if I count my previous blog posts on the subject.
I've had some false starts and gaps in my knowledge. I think that slowed me down by a factor of 2.
If I lost all data, I think I can re-do it in a week and a half.
I think we all will appreciate a follow up blog post on your experiences when you were writing it and also post-'Show HN'
Not only 0, but also some other keys listed here https://cr.yp.to/ecdh.html which produce all-zero shared key.
DJB argues you shouldn't validate public keys except for "some unusual non-Diffie-Hellman elliptic-curve protocols that need to ensure ``contributory'' behavior". So NaCl/TweetNaCl also don't do anything about it. Libsodium, on the other hand, returns an indicator when the shared key is all-zero.
One recent example where it was considered a ("low") vulnerability is in the recent wire audit: https://www.x41-dsec.de/reports/Kudelski-X41-Wire-Report-pha...
Therefore, if Bob sends all-zero ratchet public keys, subsequent message keys will only depend on the root key and not on Alice’s ephemeral private keys. A dishonest peer may therefore keep sending degenerate keys in order to reduce break-in recovery guarantees, or force all sessions initiated by a third party to use a same message key, while a network attacker may force the first message keys to a public constant value, for example.
That's said, I don't understand why DJB says this is unnecessary for Diffie-Hellman? I mean, an all-zero shared secret sounds pretty bad, doesn't it?
Or is it because DJB assumes that if one party decides to publish the communication, it can do so anyway?
In the context of most transport encryption, you presumably do want to prevent those keys from being selected by either party. That means either filtering those keys, or, perhaps better, using a fancier AKE like the 3DH in Signal/Noise or HMQV or something.
Both the API and the underlying primitives are still moving targets, but one of the motivations behind this project was to revisit the NaCl/LibSodium/SUPERCOP API to build something more difficult to misuse.
In particular, users never provide nonces. Contexts are mandatory everywhere. A broken PRG is not always catastrophic. Operations requiring keys have a dedicated keygen() function (eventually, keys may have their own type instead of generic byte buffers). There's no crypto_stream_* API, but a randombytes_buf_deterministic() function.
libhydrogen is not meant to replace libsodium, but the libsodium2 API may resemble more to it than the current one.
My library have two clear (albeit small) advantages: first, its interfaces are more consistent with its (non-existent) failure mode. NaCl and descendants have most of their function return an error code for no reason –the underlying primitives are foolproof.
Second, my Argon2i implementation doesn't allocate. Okay, the user must allocate, but that give more control, and let me guarantee I cannot fail. For some reason, the reference implementation is not like that. Surprising, considering the Argon2 authors make sure implementers could avoid spurious allocations.
---
My current recommendation still goes to TweetNaCl and Libsodium. My hope is to be able to recommend Monocypher instead. Maybe sometime later this year?
Since he's running his own server, it should be easy enough to add a certificate from https://letsencrypt.org/