Hell is overconfident developers writing encryption code | Hacker News Reader

Hell is overconfident developers writing encryption code

soatok.blog

201 pointszdw238 comments

What does "don't build your own crypto" even mean any more?

I originally thought it meant "don't implement AES/RSA/etc algorithms yourself"

But now it seems to mean "pay auth0 for your sign in solution or else you'll definitely mess something up"

As an example, we have a signing server. Upload a binary blob, get a signed blob back. Some blobs were huge (multiple GB), so the "upload" step was taking forever for some people. I wrote a new signing server that just requires you to pass a hash digest to the server, and you get the signature block back which you can append to the blob. The end result is identical (i.e. if you signed the same blob with both services the result would indistinguishable). I used openssl for basically everything. Did I roll my own crypto? What should I have done instead?

Cryptographic protocols are algorithms built on cryptographic primitives and they have plenty of footguns.

Now it's possible for someone to ask the server to sign a blob that they only know the hash of. Is that an issue in your threat model? No idea.

I feel threat modelling is the really difficult part in gluing together known-good crypto parts into a solution.

I've glued together crypto library calls a few times, and I've implemented RFCs when I've done so, like HKDF[1].

But that isn't enough if the solution I've chosen can easily be thwarted by some aspect I didn't even consider. No point in having a secure door lock if there's an open window in the back.

[1]: https://www.rfc-editor.org/rfc/rfc5869

It used to mean "use AES instead of rolling your own form of symmetric encryption." Then it became "use a library for AES instead of writing the code." It has now reached the obvious conclusion of "don't do anything vaguely security-related at all unless you are an expert."

No it hasn't. The subtext of "don't roll your own crypto" is that "AES" doesn't do, by itself, what most developers think it does; there's a whole literature of how to make AES do anything but transform 16 unstructured bytes into 16 different bytes indistinguishable from random noise; anything past that, including getting AES to encrypt a string, is literally outside of AES's scope.

The shorter way of saying this is that you should not use libraries that expose "AES", but instead things like Sodium that expose "boxes" --- and, if you need to do things that Sodium doesn't directly expose, you need an expert.

Contra what other people on this thread have suggested, reasonable security engineers do not in fact believe that ordinary developers aren't qualified to build their own password forms or permissions systems; that's a straw man argument.

Reasonable security engineers take a reasonable position on this. Many other developers (usually uninformed ones) believe this now means "don't make an auth system." Like it or not, this has become a sort of adage that people cargo-cult.

If you can't keep the line between "ordinary developers shouldn't be working with AES" and "ordinary developers can write login systems" clear, you're doing people a disservice, because the two assertions are not alike in how fundamentally true they are.

By the way, I agree with you. My original point was that the "hivemind" somehow has shifted away from the reasonable conclusion.

There's nothing personal about any of this, but I'm watching the both the insane poles of this argument ("I'll be just fine inventing my own block cipher modes" guy, "Only a certified security engineer can write a permissions check" guy") go at each other, and wondering why we need to pay any attention to either of them.

I haven't so much seen the latter kind of guy, the one saying you need a certified professional to safely output-filter HTML or whatever, but I see "lol block cipher modes whatever" people on HN all the time, on almost every thread about cryptography, dunking on anyone who says "don't roll your own cryptography".

FWIW I see a lot more of the "lol I rolled my own block cipher" crowd in articles here, but a lot more of the "only experts should verify a signature" crowd in the professional world (mostly startup-biased recently).

I sort of assume that the latter is a random sampling of software engineers but the former is sampled specifically for idiots who YOLO their crypto.

Only experts should verify signatures! That's actually hard.

If you're using minisign or Sigstore or whatever to do something like verify upstream dependencies, sure. But if you're building a system that is about some novel problem domain involving signatures: you should get an expert to verify your system. The trail of dead bodies here is long and bloody.

Yeah, coming up with your own signature scheme to verify authenticity with certain properties is very difficult but also very different in kind from running a pre-designed scheme using a library baked by someone else.

I think you have a pre-defined notion when I say the word "signature" about what I meant (perhaps something about software signing or SSL certificates). I literally meant "checking that HMAC-SHA-2 of some bytes is what you expect" (using a library for the hash). Incidentally, that's how you authenticate a JWT.

I recently came across a signature check that was (correctly) checking the signature against a public key... The issue was the public key itself was unauthenticated, and provided by the (signed) ciphertext itself... Meaning the crypto was fine, but it wasn't checking anything meaningful, as any rogue message would just include its own public key in the message!

It's not only about the raw operation of checking bytes are equal (hopefully in a constant time manner, if applicable), but also about ensuring the desired security properties are actually present in the application!

A JWT? Me me! I've seen this done wrong!

  - memcmp(actual_sig, expected_sig) == 0
  - strcmp(actual_sig_base64, expected_sig_base64) == 0
  - strcasecmp(actual_sig_base64, expected_sig_base64) == 0

> If you're using minisign or Sigstore or whatever to do something like verify upstream dependencies, sure.

No, not "sure". minisign and Sigstore are completely different things, and one needs to understand how those work and what the corresponding signatures mean: A minisign signature says "the owner of that key signed this". Whoever the owner might be at the moment of signature. Whenever that might have happened.

A sigstore signature says "Google/Github/... says that this OpenID account generated that one key that the sigstore CA attests and that signed that blob at that time". Time/ordering verification is better than in minisign, because it is there, even if through a trusted party. But identity verification relies on a trusted third and fourth party, some of whom do have a history of botching their OpenID auth.

Those are not equivalent, and knowledge is needed to not mix them up. You don't need to be an expert, but you should try to understand as much as you can, identify what you don't understand and where you must trust an expert, implementation, company, recommendation or standard to be correct and trustworthy. And then decide if that is ok with you.

You're really mining a lot out of a throwaway line there.

eh, as someone who rolls my own login systems and depends on checking salted password hashes against those in a database: Assuming I block brutes after 3 or 4 attempts, is there anything I need to worry about here besides someone getting access to the database or the salt? I think your point was that people building logins off one way hashes aren't really doing anything outrageously wrong?

I mean to start with if you are literally using "salted hashes" you are boned. There's a whole literature of "password hashes" ("password KDFs"), and that literature exists for a reason; the papers are a record of the weaknesses found in "salted hashes", a construction dating back to the late 1970s.

But once you adopt something like BCrypt or scrypt or PBKDF2 or Argon2 (literally throw a dart at a dartboard), you're into the space of things where you should, as a competent engineer, be expected to figure out a sound system on your own. The only domain-specific knowledge you really need given to you by a cryptographer is "don't just use salted hashes".

Incidentally, a presentation at the NIST crypto reading group recently pointed out some of the weaknesses of current password KDFs: Argon2i has a viable attack, bcrypt truncates input, PBKDF2 needs an utterly insane number of rounds, and scrypt and argon2d/argon2id have data-dependent computation time and seem to be vulnerable to side-channel attacks. It turns out this is a really hard problem.

I wasn't present for the NIST crypto reading group, but I documented a lot of these in 2022, if you'd like a deeper dive into that problem space: https://soatok.blog/2022/12/29/what-we-do-in-the-etc-shadow-...

None of which are material compared to someone using a salted hash.

I'm still in the dark about the problem with storing a salted hash, assuming a decent hash function and a decently long random salt.

Simply replying "everyone knows what the problem is and has know since the 70s" is not as helpful as "here is the problem in three sentences".

You can assume a base level of knowledge in your answer, as I have worked in encryption for a little bit, having written the 3DES implementation for some firmware, and done EMV development.

The very short version is that decent hash functions are very, very fast, take almost no storage to compute and can thus easily be run in parallel. If you store `(,salt ,(HMAC-SHA2-256 salt password)) in your database and it leaks, an attacker can check hundreds of millions of possible passwords every second. The password ‘open sesame’ will get found pretty early, but so will ‘0p3n#s3s4m3$’. Pretty much anything which is not of the form ‘62Jm1QyJKGIZUpb5zARRLM’ will get found pretty quickly.

You want something that is slow and takes a lot of resources to run. PBKDF2 was an early attempt which uses lots and lots of CPU, but it doesn’t use a lot of space. scrypt, bcrypt and Argon2 use lots of CPU and also lots of space in an attempt to make it more expensive to run (which is the point: you want the expected cost of finding a password to be more than the expected value of knowing that password).

That’s one level of issue.

The next level of issue is that when you run a simple system like that the user shares his password with you for you to check that it’s valid. This sounds find: you trust yourself, right? But you really shouldn’t: you make mistakes, after all. An you may have employees who make mistakes or who are malicious. They might log the password the user sent. They might steal the user’s password. Better systems use a challenge-response protocol in which you issue a one-time challenge to the user, the user performs some operation on the challenge and then responds with a proof that he knows his secret.

But those have their own issues! And then there are issues with online systems and timing attacks and more. It’s a very difficult problem, and it all matters what you are protecting and what your threat model is.

> But those have their own issues! It’s a very difficult problem, and it all matters what you are protecting and what your threat model is.

it's all a trade-off - those challenge/response systems are better, but they also have more moving parts. There's more bits in the system to go wrong.

When there's only two pieces of the system (compare salted hash of user-provided password against stored salted hash) there's very little room for errors to creep in. Your auditing will all be focused on ensuring that the user-provided password is not leaked/stored/etc after a HTTP sign-in request is rxed.

When using challenge/response, there is some pre-shared information that must synchronised between the systems (algorithm in use, key length, etc depending on the specific challenge/response system chosen). That's a great deal more points of attacks for malicious actors.

And then, of course, you need to add versioning to the system (algorithms may be upgraded over time, key lengths may be expanded, etc) that all present even more points of attack.

Compare to the simple "compare salted hash of password to stored salted hash": even with upgrading the algorithms and the key lengths there still remain only one point of attack - downloading the salted hashes!

It doesn't matter how much more secure a competing system is if it introduces, in practice anyway, more points of failure.

My takeaway after doing some cryptography for some parts of my career is that by choosing a hash function to be expensive in computational power and expensive is space and keeping the "user enters a password and we verify it" is still going to have fewer points of attack than "Synchronising pre-shared information between user and authenticator as the security is upgraded over the years".

Basically, the trade-off is between "we chose a quick hash function, but can at least upgrade everything without the client software noticing" and the digital equivalent of "It's an older code, sir, but it checks out" problems.

You keep returning to salted hashes. Please read the first two paragraphs of what I wrote for why that is a mistake. If you are going to use a shared secret system, do not use salted hashes. There is almost never a good reason to use salted hashes in 2025.

> you need to add versioning to the system

You need this with salted hashes, too! And of course with any password-based system.

> Please read the first two paragraphs of what I wrote for why that is a mistake.

Okay, I read it, and then re-read it. I still don't get why (for example) `bcrypt` (a salted hash function) is a bad idea.

I fully accept that I am missing something here, but I really would like to know why using `bcrypt` is a problem.

>> you need to add versioning to the system

> You need this with salted hashes, too! And of course with any password-based system.

Not in the client software, you don't. The pre-shared information with password-based system is generally stored in the users head.

The pre-shared information in the challenge/response system means both the submitting software (interacting with the user and rxing the challenge) as well as the receiving software (txing the challenge and rxing the response) need to be synchronised.

Now, once again, I fully accept that I might be missing something here, but AFAIK, that synchronisation contains extra points of attacks; points of attacks that don't exist in the password/salted-hash system.

And since absolutely no system ever discards existing mechanisms completely when upgrading, that deprecated but still supported for a few more months is even more additional points of attack.

Once again, I am trying to understand, not be contentious, and I want to fuolly understand:

a) The problem with salted hashes like `bcrypt`

b) What changes need to be made to client software when upgrading algorithms and key lengths in a password-based system.

bcrypt is not a "salted hash function". It's a a password hash construction (at the time of its invention, it was called an "adaptive hash"; today, we'd call it a "password KDF"). If you're using bcrypt, you're fine.

What you can't do is use SHA2 with a random salt and call it a day.

My understanding of bcrypt math is that the input to the algorithm is a random salt and a message, with the output being a hash.

I believe the actual implementation gives two output fields as a single value, with that value containing the salt and the hash.

This might be why we appear to be talking past each other - I consider bcrypt to be a salted hash because it takes a salt in the inputs and produces a hash in the output.

The fact that the output also contains the salt is, in my mind, an implementation detail.

Yes. We aren't talking past each other anymore. If you aren't composing a "salted hash" out of a cryptographic hash function and a salt, but are instead using bcrypt, scrypt, PBKDF2, or Argon2, you have nothing to worry about. Just to complete your understanding: the salt --- randomizing the hash --- has very little to do with what makes bcrypt a good password hash.

You'll avoid this confusion in the future if you don't refer to bcrypt as a "salted hash". Salted hashes are the technology bcrypt was invented to replace.

OP here: I am using bcrypt to hash passwords, but I struggle to understand how not salting i.e. not randomizing it against a stored secret would still be safe for storing short strings. Be that as it may, I'm glad to hear I'm not doing anything horribly wrong when building login systems.

In all seriousness, can you please stop recommending bcrypt for new systems? The input length is hard limited to a frighteningly short string and it is not a KDF since it has a fixed output length. Its internal state is small enough that it can be efficiently cracked with FPGAs (although the structure is unsuitable for GPU-based crackers).

The distinction between a KDF and a hash function is that a KDF has an output length that is configurable so you can directly use its output to generate a cryptographic key with 100% of the entropy of the input. A KDF theoretically needs to have arbitrary input length and arbitrary output length. Bcrypt has neither.

Argon2d is great, scrypt is great, PBKDF2 with 1000000+ rounds works fine if you want FIPS, but bcrypt is somehow still on many people's list as though it doesn't have these flaws. It's not at the point where it needs to be imminently replaced in a legacy system, but if you're building something new, you should use something better.

Mostly, here in 2025 the question is: "Why are you using shared secrets?" - which is why this discussion comes up. These are never about a PAKE or the PIN for a Yubikey or something, it's always shared secrets when people say "passwords".

This is a terrible stopgap solution, in 1975 it's reasonable to say we have other priorities right now, we'll get to that later. But it's very silly that in 2025 you're telling people to try PBKDF2 when really they shouldn't even be in this mess in the first place.

Remember when Unix login still hadn't solved this? 1995. That was last century. And yet, here we are, still people are writing "Password" prompts and thinking they're doing a good job.

By salted hashes, I'm pretty sure OP meant using something like bcrypt or PBKDF2

Thanks, yes, using bcrypt. Usually just with something like PHP's password_hash() and the default algorithm. I sort of expected to trigger people with my post, and it's enlightening, but I feel pretty safe with what I'm doing. I mean, I like to know exactly how everything in my engine works. But there's a point at which I can't get paid to reinvent the wheel anymore, nor do I want to try.

You'll want to look at your hashing algorithm and number of iterations to make sure they are up to snuff and you'll want to make sure that your salts are unique and properly random. (I recently learned that the PS3 private keys were reversible because Sony's random number function returned a fixed result.)

Proper password hashing is protecting against a different threat than blocking brute force login attempts (which isn't really an encryption issue.) Password hashing is more about protecting your user's passwords if your database is breached so that you limit the attackers ability to re-user those credentials to access other services the user may have re-used them with.

It's complicated, no?

Reasonable developers are qualified to do those things. But to build a full-featured authentication subsystem for their webapp? If it's something that holds any kind of reasonably private info, I'm not so sure.

Sure, a reasonable developer will use some sort of PBKDF to hash passwords. But when users need a password reset over email, will they know not to store unhashed reset tokens directly in the database? Will they know to invalidate existing sessions when the user's password is reset? Will they reset a browser-stored session ID at login, preventing fixation? And on and on and on. The answer to some of these questions will be yes, but most developers will have a few for which the answer is no. Hell, I've probably built more auth systems than most (and have reported/fixed a few vulnerabilities on well-known open-source auth systems to boot) and I'm honestly not sure I'd trust myself to do it 100% correctly for a system that really mattered.

Even outside of "holding the crypto wrong", these things have sharp edges and the more you offload to an existing, well-vetted library the more likely you are to succeed.

There are different levels of security. The problem with confidently incorrect level is that it's worse than nothing. Having plaintext tokens (or even passwords) in a DB has its risks, but those risks are (a lot more) obvious, they are not shadowy bugs lying in wait under heaps of code.

Based on breaches in systems like Okta and others, I don't think anyone knows how to do it 100% correctly.

Or the companies have figured out it's more expensive to hire a full time professional cryptographer to vet the process, than to deal with the PR for now more or less normalized data breaches.

Having worked at a competitor, I’ll simply say I know nothing specifically of Okta’s hiring practices, but it’s bold of you to assume they’re hiring any security engineers whatsoever. Still, this was twelve years ago when I was at the competitor in question so all the companies involved were was much smaller than they are today. But quite literally zero people at the particular competitor knew or even particularly cared about security.

Proving your point, but: What is fixation?

https://en.wikipedia.org/wiki/Session_fixation

In short, in at least one variation, the attacker is able to smuggle in a known (unauthenticated) session token into the victims browser. Once the victim logs in the session token is authenticated and known to the attacker.

The easy countermeasure is to renew the session token on login and not reuse a previously unauthenticated session token. Or your application has no session at all before login.

I am not a cryptography expert but even I know it's wrong that a company I worked at stored passwords in plaintext because my boss told me, customers call in to ask for their password and supports happily gives it to them.

When I rolled my eyes up, I soon got a termination.

Unfortunately not really.

Amount of foot guns in auth flow is high. Implementation of login / password form is just a small piece.

Making sure there are no account enumeration possibilities is hard. Making sure 2FA flow is correct and cannot be bypassed is hard. Making proper account recovery flow has its own foot guns.

If you can use off the shelf solution where someone already knows about all those - it still stands don’t roll your own.

There's even bogus "But it's encrypted" crypto in the new software I'm working with in my day job this year 2025. There are so many other "must fix" user visible problems that I actually forgot to mention "Also the cryptography is nonsense, either fix it or remove it" on the one hour review meeting last week, but I should add that.

I've made one of these goofs myself when I was much younger (32 byte tokens, attacker can snap them into two 16-byte values, replace either half from another token and that "works" meaning the attacker can take "Becky is an Admin" and "Jimmy is a Customer" glue "y is an Admin" to "Jimm" and make "Jimmy is an Admin") which got caught by someone more experienced before it shipped to end users, but yeah, don't do that.

The stuggle was real in the early web with getting companies to do proper password storage and authentication but the fact that seasoned professionals turn to auth0 or okta (and have been bitten by this reliance!) nowadays strikes me as a little embarrassing.

Let's be fair though: Auth0 does a little more than just password authentication

Registration, 2FA, reset, email verification, federation, password rules, brute force protection, RBAC/ABAC etc

(I'm no fan of Auth0 fwiw)

And the consequence is that people get banged on the head if they either use sth existing (cause they will be using it wrong) or they build sth on their own (because that's obviously bad) or they get fed up and don't use anything.

The issue with security researchers, as much as I admire them, is that their main focus is on breaking things and then berating people for having done it wrong. Great, but what should they have done instead? Decided which of the 10 existing solutions is the correct one, with 9 being obvious crap if you ask any security researcher? How should the user know? And typically none of the existing solutions matched the use case exactly. Now what?

It's so easy to criticize people left and right. Often justifiably so. But people need to get their shit done and then move on. Isn't that understandable as well?

> The issue with security researchers, as much as I admire them, is that their main focus is on breaking things and then berating people for having done it wrong.

This is plain incorrect in my experience.

Recommended reading (addresses the motivations and ethics of security research): https://soatok.blog/2025/01/21/too-many-people-dont-value-th...

> Great, but what should they have done instead? Decided which of the 10 existing solutions is the correct one, with 9 being obvious crap if you ask any security researcher?

There's 10 existing solutions? What is your exact problem space, then?

I've literally blogged about tool recommendations before: https://soatok.blog/2024/11/15/what-to-use-instead-of-pgp/

I'm also working in all of my spare time on designing a solution to one of the hard problems with cryptographic tooling, as I alluded to in the blog post.

https://soatok.blog/2024/06/06/towards-federated-key-transpa...

Is this not enough of an answer for you?

> How should the user know? And typically none of the existing solutions matched the use case exactly. Now what?

First, describe your use case in as much detail as possible. The closer you can get to the platonic ideal of a system architecture doc with a formal threat model, the better, but even a list of user stories helps.

Then, talk to a cryptography expert.

We don't keep the list of experts close to our chest: Any IACR-affiliated conference hosts several of them. We talk to each other! If we're not familiar with your specific technology, there's bound to be someone who is.

This isn't presently a problem you can just ask a search engine or generative AI model and get the correct and secure answer for your exact use case 100% of the time with no human involvement.

Finding a trusted expert in this field is pretty easy, and most cryptography experts are humble enough to admit when something is out of their depth.

And if you're out of better options, this sort of high-level guidance is something I do offer in a timeboxed setting (up to one hour) for a flat rate: https://soatok.com/critiques

> I've literally blogged about tool recommendations before

Do you happen to know of a similar resource applicable to common HN deployment scenarios, like regular client-server auth?

For example, in your Beyond Bcrypt blog post[0] you seem to propose hand-writing a wrapper around bcrypt as the best option for regular password hashing. Are there any vetted cross-language libraries which take care of this? If one isn't available, should I risk writing my own wrapper, or stick with your proposed scrypt/argon2 parameters[1] instead? Should I perhaps be using some kind of PAKE to authenticate users?

The internet is filled with terrible advice ("hash passwords, you can use md5"), outdated advice ("hash passwords, use SHA with a salt"), and incomplete advice ("just use bcrypt") - followed up by people telling you what not to do ("don't use bcrypt - it suffers from truncation and opens you up to DDOS"). But to me as an average programmer, that just leave behind a huge void. Where are the well-vetted batteries-included solutions I can just deploy without having to worry about it?

[0]: https://soatok.blog/2024/11/27/beyond-bcrypt/

[1]: https://soatok.blog/2022/12/29/what-we-do-in-the-etc-shadow-...

Your tool recommendations are actively harmful and dangerous in places.

You whole-heartedly recommend sigstore, a trusted-third-party system which plainly trusts the auth flows of the likes of Google or Github. It is basically a signature by OpenID-Login. This is no better than just viewing everything from github.com/someuser as trusted. The danger of key theft is replaced by the far higher danger of account theft, password loss and the usual numerous auth-flow problems with OpenID.

Why should I take those recommendations seriously?

I feel like there is a huge subset of people who just do trial and error programming. The other day, I watched a whole team of 8 people spend a whole work day swapping out random components on Zoom to diagnose a problem because not a single person considered attaching a debugger to see the exact error in 5 minutes.

I feel like you have to tell people to not roll your own whatever because there are so many of these types of people.

I was so surprised the first time I saw this. I mostly work alone and my workflow is usually study an example (docs or project), collect information (books and article), and iteratively build a prototype while learning the domain. Then I saw a colleague literally copying and pasting stuff, hoping that the errors go away in his project. After he asked for my help, I tell him to describe how he planned to solve the problem and he couldn't. For him, it was just endless tweaking until he got something working. And he could have understood the (smallish) problem space by watching one or two videos on YouTube.

Those same people are utterly incapable of reading logs. I’ve had devs send me error messages that say precisely what the problem is, and yet they’re asking me what to do.

The form of this that bothers me the most is in infra (the space I work in). K8s is challenging when things go sideways, because it’s a lot of abstractions. It’s far more difficult when you don’t understand how the components underpinning it work, or even basic Linux administration. So there are now a ton of bullshit AI products that are just shipping return codes and error logs out to OpenAI, and sending it back rephrased, with emoji. I know this is gatekeeping, and I do not care: if you can’t run a K8s cluster without an AI tool, you are not qualified to run a K8s cluster. I’m not saying don’t try it; quite the opposite: try it on your own, without AI help, and learn by reading docs and making mistakes (ideally not in prod).

> What should I have done instead?

Security has largely to do with trust.

When asked who I trust most in this space, the answer is always libsodium.

I leave as much of the protocol as possible to their implementation.

https://doc.libsodium.org/

The sad truth is that even libsodium can be misused (and the article explicitly mentions that twice). Even pretty high-level constructions like libsodium's crypto_secretbox can be misused, and most of the constructs that libsodium exposes are quite low-level: crypto_sign_detached, crypto_kdf_hkdf, crypto_kx, crypto_generichash.

I think we should understand and teach "do not roll your own crypto" in the context of what constructs are used for what purposes.

AES is proven to be secure and "military-grade" if you need to encrypt a block of data which is EXACTLY 128 bit (16 bytes) in size. If you want to encrypt more (or less) data with the same key, or make sure that the data is not tampered with or even just trust encrypted data that passes through an insecure channel, then all bets are off. AES is not designed to help you with that, it's just a box that does a one-off encryption of 128 bits. If you want do anything beyond that, you go into the complex realm of chaining (or streaming) modes, padding, MACs, nonces HKDFs and other complex things. The moment you need to combine more than two things (like AES, CBC, PKCS#7 padding and HMAC-SHA256) in order to achieve a single purpose (encrypting a message which cannot be tampered), you've rolled your own crypto.

Libsodium's crypto secretbox is safe for encrypting small or medium size messages that will be decrypted by a trusted party that you share a securely-generated and securely-managed key with. It's far more useful than plain AES, but it will not make any use case that involves "encryption" safe. If you've decided to generate a deterministic nonce for AES, or derive a deterministic key (the article links a good example[1]), then libsodium will not protect you. If you attempt to encrypt large files with crypto_secretbox by breaking them to chunks, you will probably introduce some vulnerabilities. If you try to build an entire encrypted communication protocol based on crypto_secretbox, then you are also likely to fail.

The best guideline non-experts can follow is the series of Best Cryptographic Answers guides (the latest one I believe is Latacora 2018[1]). If you have a need that is addressed there and you only need to use the answer given with combining it with something else, you're probably safe.

Notice that the answer for Diffie-Hellman is "Probably nothing", since you're highly unlikely to be able to use key exchange safely in isolation. I did roll key exchange combined with encryption once, and I still regret it (I can't prove that thing is safe).

The "You care about this" part explains the purpose of each class of cryptographic constructs that has a recommendation. For instance, for asymmetric encryption it says "You care about this if: you need to encrypt the same kind of message to many different people, some of them strangers, and they need to be able to accept the message asynchronously, like it was store-and-forward email, and then decrypt it offline. It’s a pretty narrow use case."

So this tells you that you probably should not be using libsodium's crypto_box for sending end-to-end encrypting messages in your messaging app.

[1] https://www.latacora.com/blog/2018/04/03/cryptographic-right...

[1] https://www.cryptofails.com/post/75204435608/write-crypto-co...

I honestly think it's a meme born of propaganda. 'Don't roll your own' is good advice a lot of the time when there are hardened libraries at your disposal, but who does it serve if less people know how to build useful cryptography? Three letter agencies mostly. I had no trouble implementing useful algorithms in college as part of my study of it, or understanding different potential weaknesses like timing or known plaintext attacks. We didn't cover side channels like power analysis but how often does that actually matter outside of a lab?

No, you can not competently roll your own crypto because you took an algorithms class in college. Even the assumption you could is absurd.

It wasn't 'an algorithms class' but grad level cryptography. We did implement boxes as a project and were graded on them. Where do you think people learn this stuff? Super secret cryptography school? I even used it. I previously built a secondary layer protocol using diffie hellman that runs on top of TLS such that in the event TLS is broken there's still some level of secrecy, and I'm very confident that it's better than nothing. Defense in depth, you know?

Yes, most of us have likely taken a grad level cryptography course. Still, that's not enough to securely roll your own crypto.

You want many years of experience and a community of experts working with you. If you don't have that - 99.9999% of devs do not - just use libsodium.

The arrogance and elitism is wild. I never said people shouldn't use libsodium.

No, instead you said you learned about how to understand "known plaintext attacks", counting on the average reader of this thread not to know that's a content-free claim. One time, in a "graduate level" cryptography class, you even built a protocol using Diffie-Hellman on top of TLS. OK!

That project wasn't for school.

You provided it as a proof-point of some sort, but I've led lab exercises of freshmen CS students doing the same exercise, so I'm unclear what it's proof of.

Proof that you don't need to be super-human to build something useful, also demonstrated by your freshman CS student anecdote.

Anybody can build a Diffie-Hellman protocol. You can practically do with a calculator; in fact, we did that as an exercise at a talk, with a big audience, using frisbees to "exchange" the keys. But: the talk was about how, as a pentester, to trivially break these systems, because making a system that uses a DH key exchange safely, as opposed to one that simply appears correct in unit tests, is treacherously hard.

I'm still curious about what you could have possibly meant by learning about "weaknesses" like "known-plaintext attacks". Can you say more?

It's been years, but I recall that, for example, when you know every piece of plaintext starts with "https://www", or perhaps know the full contents of particular messages, it may become some degree easier to brute force your way to a key. I don't think it's a concern for standards in broad use, more like something you would worry about if you were cooking up your own cipher.

It isn't arrogant or elitist to suggest that no one person should be rolling their own crypto, even if they have taken a grad level cryptography class.

For instance, you don't expect an aviation engineer to build a brand new plane on their own. They would be missing decades of cumulative knowledge, battle testing, perspectives and knowledge outside of their own. These systems are complex, to the point where taking a grad course is not enough.

I have worked with actual experts on cryptography throughout my big tech career - people that have actually written parts of common crypto libraries suggested in this thread - and even they themselves are not interested in writing crypto code. It is an incredibly involved group effort between experienced experts and your first iteration will almost certainly be broken. There is almost never a reason to do this.

If you are so confident in your ability to do so, you may simply be a crypto prodigy, and I apologize. You should post your DHE implementation here. If it's secure and useful, there shouldn't be an issue, and surely the community would benefit from it.

Again you're mistaking my point. I know enough to know what I don't know and the risk involved in doing any serious crypto work, that I can't solo build libraries to protect financial transactions or whatever willy-nilly. It's obviously a ton of work to get right.

What I'd advocate for instead of "never roll your own crypto" is more like "never use your own crypto in prod". People are better off knowing more than less and getting their hands a little dirty. I think the former, common message is more like "don't even try to understand it," which is a joke.

The "in prod" is implicit, no one is trying to stop you from learning things.

It makes more sense if you understand that gatekeeping like this also provides job security for security consultants telling you to buy their services instead.

The fact that you're referring to useful algorithms here is kind of a "tell", for what it's worth. "Known plaintext attacks" are not a broad class of vulnerability in cryptosystems; they're a setting for attacks. It's like saying "I've studied the browser security model including weaknesses such as the same origin policy".

What do you think an algorithm is? This weird criticism is kind of a "tell", for what it's worth. Ciphers are algorithms. This is unnecessarily combative. I'm disappointed by your posts in this thread, because they're low on substance an high on posturing.

Learning crypto isn’t just a box ticking exercise that you learn it and now you’re qualified to “do crypto” for the rest of your career. I also studied this in university but I know that I don’t spend anywhere near enough time to day to avoid, well, any pitfalls of rolling my own crypto.

I currently work on our company’s Auth systems and half my team still side-eyes our own work, despite having worked on this area for years now.

All I'm reading here is more of the same scare-mongering. I don't casually implement cryptosystems to be used in critical situations, and I wouldn't attempt it without a serious amount of research. But "never do it yourself" is a poor substitute for actually spreading knowledge about how to do it and open discussion of pitfalls. It rubs me the wrong way.

You do understand “never do it yourself” implicitly applies to a commercial/production context, right? Nobody is stopping you learning or choosing to enter the field (to make it your sole professional focus).

It certainly comes across like people are discouraging others from even learning about it.

Just read through this thread and see how few people are saying anything like "you should still learn how cryptographic primitives work though". It's almost nothing but negativity and discouragement.

The blog post in question (which I wrote) literally opens with a link to https://www.cryptofails.com/post/75204435608/write-crypto-co... that encourages people to learn about it! Yes, even if that type of learning means (privately) writing crypto code.

Why would you expect the comments about it to argue in favor of "still learning" when the blog post never advocated against learning? They aren't making this point because I didn't make the contrary one, so there's nothing to argue with.

That you're only seeing negativity and discouragement is a cognitive distortion. I'm not qualified to help people with those, but I hope pointing that out might help you see that the world isn't against you.

That software developers presently and generally are bad at understanding cryptography doesn't mean it's impossible to learn, or that you should be discouraged from learning.

Alright that's fair, in this thread there's not much reason for people to encourage learning it so it wasn't a good example to use. Thank you for helping people to learn more about it.

That being said, the sentiment around rolling your own crypto that I see in the rest of internet in general still strongly discourages any kind of curiosity, intentionally or not.

In other places where it's discussed and in the comments on this thread which are not a direct response to aspects of the blog, you get mostly the same kind of response.

> Just read through this thread and see how few people are saying anything like "you should still learn how cryptographic primitives work though". It's almost nothing but negativity and discouragement.

That’s what the “implied” part of my comment means.

With signing server level of interaction is the thing that really matters. I would probably say that if you implemented algo doing the signing operation it might be questionable. But if you implemented the key storage fully yourself, it is even more questionable. Or design your HSM module, or just write the keys in plaintext or minimally protected on the disk...

Also who designed and implemented checking the appended blob? Also crypto...

It never meant "don't implement AES/RSA yourself". Nobody sane does that; there has never been a need to convince people not to write their own implementations of AES. Ironically, doing your own AES is one of the less-scary freelancing projects you can undertake. Don't do it, but the vulnerabilities you'll introduce are lower-tier than the ones you get using OpenSSL's AES.

It has, always, mean "don't try to compose new systems using things like AES and RSA as your primitives". The serious vulnerabilities in cryptosystems are, and always have been, in the joinery (you can probably search the word "joinery" on HN to get a bunch of different fun crypto vulnerabilities here I've commented on).

Yes: in the example provided, you rolled your own cryptography. The track record of people building cryptosystems on top of basic signature constructions is awful.

I assume you've never seen AES written in vbscript? Generally, the thought process goes: This thing needs AES, I want to talk to it, I know $language, and wikipedia has the algorithm. The idea that AES is there for a reason ( like security) never enters the thought process.

Someone is building a shed, not a whole building, and stopped listening to real builders with their nitpicky rules long ago. It works great, until the shed has grown into a skyscraper without anybody noticing, and an unexpected and painfull lesson about 'load bearing' appears.

I ran the Cryptopals challenges. I have been sent AES implemented in 4 different assemblies, Julia before it was launched, pure Excel spreadsheet formulae, and a Postscript file.

There must be beauties in there;-). Even so, the fact that it's called Cryptopals indicates a public having some basic level of care about crypto. The non-it person hacking an excel macro together to get some job done has a very different attitude, and they do run their stuff in production.

The public should have a basic level of understanding and care of crypto. The logic of those challenges is that encouraging people to break cryptography is always prosocial; building it is a little more complicated, in the same sense as surgery.

So... you've pointed out the problems, what are the solutions?

What am I allowed to use as primitives to compose systems that require cryptographic functionality? If I'm writing medical device software and the hospitals I'm selling to say I can't store files in plaintext on disk, but also some security expert on HN says I shouldn't use AES as a piece of the solution because that's "rolling my own crypto" and too dangerous, what should I do? Mandate that postgres configured with encryption be used (even if the application is simple and doesn't require a full db)? That will almost certainly harm the prospect of the sale because having to get hospital IT involved introduces a lot of friction. Or are you saying "use sodium to encrypt the file, don't try to do it yourself with AES"?

> What am I allowed to use as primitives to compose systems that require cryptographic functionality?

The problem isn't the primitives, it's the act of a custom composition.

The problem isn't whether AES is used. The problem is whether you're writing code that interfaces at the level of 128-bit blocks.

Want a canned solution for generic problems?

https://soatok.blog/2024/11/15/what-to-use-instead-of-pgp/

Want a specific solution for your specific use case? Talk to an expert to guide you through the design and/or implementation of a tool for solving your specific problem. But don't expect everyone writing for a general audience (read: Hacker News comments) to roll out a bespoke solution for your specific use case.

There's a subtext here of "what do I do when the high-level libraries like Sodium don't do exactly what I need", and the frank answer to that is: "well, you're in trouble, because consulting expertise for this work is extraordinarily expensive".

We have an SCW episode coming out† next week about cryptographic remote filesystems (think: Dropbox, but E2E) with a research team that reviewed 6 different projects, several of them with substantial resources. The vulnerabilities were pretty surprising, and in some cases pretty dire. Complaining that it's hard to solve these problems is a little like being irritated that brain surgery is expensive. I mean, it's good to want things.

† https://securitycryptographywhatever.com/

> There's a subtext here of "what do I do when the high-level libraries like Sodium don't do exactly what I need", and the frank answer to that is: "well, you're in trouble, because consulting expertise for this work is extraordinarily expensive".

...which is exactly why people roll their own crypto. Security folks don't seem to realize/care that money is a real constraint at most companies (esp. startups) and security can easily become a money furnace. When the only two options are: "use sodium" and "extraordinarily expensive consultant" then devs turn to the third option: https://pkg.go.dev/crypto/aes

Boom, file encrypted, requirement fulfilled, money saved, sale made. Perfectly secure? Probably not. The docs even say "The AES operations in this package are not implemented using constant-time algorithms". But maybe that's an acceptable tradeoff for your target risk profile.

If what you need is filed stored encrypted on disk... why is "use sodium" not a viable option for you?

It might be in many cases, but not every language it seems has explicit sodium support. Golang for example, has this not-widely-used wrapper (https://github.com/jamesruan/sodium) maintained by some random guy which might not be desirable to use for a variety of reasons.

Foreign function interfaces aren't that scary. Any competent $lang developer ought to be able to read the docs and roll their own wrapper.

I bet more than 90% of the devs turning to a third option could in fact use sodium or an equivalent. Especially if the goal is "boom, file encrypted".

> or an equivalent

How does one identify the proper "equivalent" in a given programming language that doesn't have 1st class sodium support?

Second class sodium is still a hell of a lot better than setting up your own. For go in particular, writing your own wrapper is a reasonable option. It's a handful of lines of code per function used.

I don't have a full guide for identifying a proper equivalent, but "constant time" is a requirement.

I want to roll my own variant of AES (I know, I know!) CTR mode for appendable (append-only) files that does not require a key change or reencrypting any full AES block. Big caveat, this design doesn't have a MAC, with all the associated problems (it's competing against alternatives like AES-XTS, not authenticated modes).

Partial blocks are represented by stealing 4 bits of counter space to represent the length mod block size. This restricts us to 2^28 blocks or about 4GB, but that's an ok limitation for this use.

So say you initially write a file of 33 bytes: two full blocks A and B, and a partial block C. A and B get counter values 0 (len) || 0 (ctr) and 0 (len) || 1 (ctr). C is encrypted by XORing the plaintext with AES(k, IV || 1 (len) || 2 (ctr)).

You can append a byte to get a length of 34 bytes. Encrypted A/B don't change. C_2 is encrypted by XORing plaintext_2 with AES(k, IV || 2 (len) || 2 (ctr)). Since the output of AES on different inputs is essentially a PRF, this seems... ok?

Finally if you append enough bytes to fill C, it gets to len=0 mod 16. So the long and short if it is: no partial or full block will ever reuse the same k+iv+len+ctr, even rewriting it for an append.

> I want to roll my own variant of AES (I know, I know!) CTR mode for appendable (append-only) files that does not require a key change or reencrypting any full AES block.

https://xyproblem.info

Why, exactly, do you want to do that at all?

I want the contents to be unreadable to an attacker who is able to steal a disk. I want to be able to append to files but not overwrite them in place, and do partial reads without decrypting the entire file.

(Google's AES-XCTR / HCTR2 seems somewhat similar.)

Hmm, are you familiar with NIST's workshops on accordion cipher modes?

I would talk to the people involved in that effort rather than wholesaling a design from scratch.

Interesting!

Building crypto systems is also hard.

You write about passing hash digest.

Makes me think about pass the hash vuln in windows NTLM where if someone grabs the hash they don’t need to know the pass anymore because they can pass the hash.

The same you still can use AES or RSA incorrectly so that whatever you built on top of those is vulnerable if you don’t have experience.

OK TFA explains all I have the same view on topic as author.

building your own crypto is so simple a 5 year old can do it, just roll a dice, and write down the numbers, those numbers are the key. Then convert the text to numbers, and for each letter add the corresponding key number, and the sum will be the encrypted text. To decrypt you subtract the number with the corresponding number on the key. The problem is that modern crypto is so complicated one will easily make an error implementing it.

That works if your key is long as heck, or the message is short as heck. Still, a useful way to quietly entertain (some) kids for a little while.

In that scheme -5 is always A and there are only 6 options for any particular letter, so you can do a slightly more complicated frequency analysis.

Sounds like it might be good for kids learning how to break crypto?

Add to encrypt, so 32 is always Z.

It's unbreakable, google for "one time pad" encryption. The problem with cracking it is that if you just change one letter in an English word it becomes another English word. Here i have encrypted "hackernews" with the following key 2,3,1,6,2,1,2,2,1,3 Good luck cracking it without the key: JDDQGSPGXV

As was said above, don't just add a number from 1 to 6. You want your ciphertext to be indistinguishable from random, so in this case (assuming only letters and no punctuation nor space), you need to add a number from 0 to 26. And you need a uniform distribution, or again, statistical analysis will screw you.

Now one does not simply generate a uniform random distribution from a d6. You can't throw the die 5 times and add the results for instance. First, you'll get a number between 5 and 30. Not only the range falls short one letter (25 possibilities instead of 26), what you get is a binomial distribution, heavily skewed towards the middle numbers (5 and 30 are very improbable in comparison).

My recommendation here would be to throw the die once, subtract 1, multiply by 6, throw it a second time, add it, subtract 1. This will give you a uniform distribution between 0 and 35 (assuming your die is perfectly fair, which, spoiler alert, it isn't). Think of it of a 2-digit base 6 number. Assign a number to each letter of the alphabet (and now you can even support spaces & punctuations, or numbers), add (modulo 35) the result of your die throw, do not translate the result to an encoding with less than 36 symbols, and now you have a proper one time pad.

One thing that's crucial when attempting anything cryptographic: make sure you've got the maths down. Ideally you should be able to construct a rigorous (even machine verified) proof that your stuff works as intended. For the one time pad it's relatively easy. But you need to do it, that's how you'll notice that if you encrypt "aaaaaaaaa" your result will only yield letters between B and G, which you'll agree doesn't hide your plaintext nearly as well as you intended.

There's a load of places you can shoot yourself in the foot, e.g., not using constant time comparison functions, which leaves you open to timing attacks.

Glueing together secure pieces too often does not result in a secure whole. People spend careers coming up with secure protocols out of secure pieces.

My take on not rolling your crypto is to apply as close to zero cleverness as possible when it comes to crypto. Take ready made boxes, use them as instructed and assume that anything clever I try to build with them is likely not secure, in some way or another.

This is a relatively long post that is kind of beating around the bush: these developers believed that OpenSSL was a trustworthy substrate on which to build a custom cryptosystem. It's not; this is why libraries like Tink and Sodium exist. They don't really need much more lecturing than "stop trying to build on OpenSSL's APIs."

tptacek I don't want to waste your time, but do you have any good recommendations for material that bridges the gap between modern deployed cryptosystems and the current SOTA quantum computing world in enough detail that is useful for an engineer practitioner preparing for next 10 years?

Nope. It's at times like these I'm glad I've never claimed I was competent to design cryptosystems. I'm a pentester that happens to be able to read ~30% of published crypto attack papers. My advice is: ask Deirdre Connolly.

My standard answer on PQC about about the quantum threat is: "rodents of unusual size? I don't think they exist."

I have become a bit of a cryptographer (after running a cryptography-related company for a while), and aside from joke thought experiments, I am one of the most conservative cryptographic programmers I know.

I'm personally pretty skeptical that the first round of PQC algorithms have no classically-exploitable holes, and I have seen no evidence as of yet that anyone is close to developing a computer of any kind (quantum or classical) capable of breaking 16k RSA or ECC on P-521. The problem I personally have is that the lattice-based algorithms are a hair too mathematically clever for my taste.

The standard line is around store-now-decrypt-later, though, and I think it's a legitimate one if you have information that will need to be secret in 10-20 years. People rarely have that kind of information, though.

I agree. (I'm a quantum skeptic personally.)

> I'm personally pretty skeptical that the first round of PQC algorithms have no classically-exploitable holes

I was of the impression that this was the majority opinion. Is there any serious party that doesn't advocate hybrid schemes where you need to break both well-worn ECC and PQC to get anywhere?

> The standard line is around store-now-decrypt-later, though, and I think it's a legitimate one if you have information that will need to be secret in 10-20 years. People rarely have that kind of information, though.

The stronger argument, in my opinion, is that some industries move glacially slow. If we don't start pushing now, they won't be any kind ready when (/if) quantum computing attacks become feasible. Take industrial automation: Implementing strong authentication / integrity protection, versatile authorization and reasonable encryption into what would elsewhere be called IoT is just now becoming an trend. State-of-the-art is still "put everything inside a VPN and we're good". These devices usually have an expected operational time of at least a decade, often more than one.

To also give the most prominent counter argument: Quantum computing threats are far from my greatest concerns in these areas. The most important contribution to "quantum readiness"[1] is just making it feasible to update these devices at all, once they are installed at the customer.

[1] Marketing is its own kind of hell. Some circles have begun to use "cyber" interchangeable with "IT Security" – not "cyber security" mind you, just "cyber".

Yes: there are reasonable, reputable cryptographers who advocate against hybrid cryptosystems.

Could you be so kind to provide a link or reference? I'd like to read their reasoning. Given the novelty of e.g. Kyber, just relying on it alone seems bonkers.

No. I don't agree with them.

The post links to this GitHub issue [1] where the critic explains his issues with the design and the programmer asks him to elaborate on how those crypto issues apply to his implementation. The critic's reply does not convince me. It doesn't address any points, and refers to some vague idea about "boring cryptography". In what way is AWS secrets manager or Hashicorp Vault more "obviously secure" than the author's 72-line javascript file?

[1] https://github.com/gristlabs/secrets.js/issues/2

And the critic's only argument is a link to their own blog...

Those aren’t even the correct answer for the use-case in question, anywho. What they’re looking for would actually be sops (https://github.com/getsops/sops), or age (made by the fantastic Filo Sottile: https://github.com/FiloSottile/age), or, hell, just using libsodium sealed boxes. AMS KMS or Vault is perhaps even worse of an answer, Actually

>It doesn't address any points

Taking some time to point out the vulnerability is already charity work. Assuming that's also a commitment to a free lecture on how the attacks work, and another hour of free consultation to look into the codebase to see if an attack could be mounted, is a bit too much to ask.

Cryptography is a funny field in that cribs often lead to breaks. So even if the attack vector pointed out doesn't lead to complete break immediately, who's to know it won't eventually if code is being YOLOed in.

The fact the author is making such a novice mistake as unauthenticated CBC, shows they have not read a single book on the topic should not yet be writing cryptographic code for production use.

> Taking some time to point out the vulnerability is already charity work

Sure, but if you’re not going to reason why the vulnerability you’re pointing out is an issue or respond well to questions then it’s almost as bad as doing nothing at all.

A non expert could leave the same Maintainers on many Github pages. Developers can’t be expected to blindly believe every reply with a snarky tone and a blog link?

If the snarky comment is "your crypto implementation is bad", then, yes, I would always take that seriously. If I really know what I'm doing then I'll be able to refute the comment; if not, then I probably should be using an audited library anyway.

Mistakes in crypto implementation can be extremely subtle, and the exact nature of a vulnerability difficult to pin down without a lot of work. That's why the usual advice is just "don't do it yourself"; the path to success is narrow and leads through a minefield.

It probably is stemming from the fact that cryptography needs to be perfect code to guarantee total confidentiality with complete integrity. Without this goal of writing perfect code, that always encrypts and decrypts, but only for the keyholder(s), they're simply begging to get things wrong and waste time.

>Developers can’t be expected to blindly believe every reply with a snarky tone and a blog link?

Developers are adults with responsibility to know the basics of what they're getting into, and you don't have to get too far into cryptography to learn you're dealing with 'nightmare magic math that cares about the color of the pencil you write it with', and that you don't do stuff you've not read about and understood. Another basic principle is that you always use best practices unless you know why you're deviating.

The person who replied to that issue clearly understands some of the basics, or they at least googled around, since they said "Padding oracle attacks -- doesn't this require the ability to repeatedly submit different ciphertext for decryption to someone who knows the key?"

In what college course or book is padding oracle described without mentioning how it's mitigated, I have no idea. Even Wikipedia article on padding oracle attacks says it clearly: "The CBC-R attack will not work against an encryption scheme that authenticates ciphertext (using a message authentication code or similar) before decrypting."

The way security is proved in cryptography, is often we give the attacker more powers than they have, and show it's secure regardless. The best practices include the notion that you do things in a way that categorically eliminates attacks. You don't argue about 'is padding oracle applicable to the scenario', you use message authentication codes (or preferably AE-scheme like GCM instead of CBC-HMAC) to show you know what you're doing and to show it's not possible.

If it is possible and you leave it like that because the reporter values their time, and they won't bother, an attacker won't mind writing the exploit code, they already know from the open source it's going to work.

How is it as bad if no signal means you'll likely never fix it, and some signal means you're more likely to fix it? Like seriously, this is the only 1 (one!) issue in this repo, not hundreds of bot vulnerability submissions, where is this fear of snarky replies come from?

> Developers can’t be expected to blindly believe every reply with a snarky tone and a blog link?

Sure, they can google around, read the blog, do other steps to educate themselves on crypto - all with their eyes wide open - before realizing they've made a big mistake and fixing vulnerabilities (and thanking the snarky author for the service)!

Wow, the smugness of that reply. Responding by calling someone naive and blowing them off despite there being real questions.

The “insecure crypto “ that they clearly link to (despite not wanting to put them on blast) was also a bit overdone. I guess we all are stuck hiring this expert to review our crypto code(under NDA of course) and tell us we really should use AWS KMS.

AWS KMS is great product branding. I've never seen another company so accurately capture how it feels to use their product with just the name before.

It's also just a profoundly good product. If you can use KMS, you should.

Always be suspicious of any acronym with a ‘K’ in it, just on general principle.

There are some weird attacks against KMS that I think are possible that are not obvious. For example KMS has a mode where it will decrypt without supplying a key reference (suspicious!). If an attacker can control the cipher text then they can share a KMS key from their AWS account to yours and then control the plaintext. I haven’t confirmed this works so maybe my understanding is incorrect.

Also, with KMS you probably should be using the data key API but then you need some kind of authenticated encryption implemented locally. I think AWS has SDKs for this but if you are not covered by the SDK then you are back to rolling your own crypto.

I agree with his comment and would like to add that the critic came across as rude and superior. Instead of answering the dev’s question in good faith, they linked to their own blog entry that has the same tone. Is it a cryptographic expert thing to act so rude?

Great question. AWS secrets and Hashicorp Vault have both been audited by a plethora of agencies (and have passed). GitHub code for someone's pet project very likely isn't going to pass any of those audits. When something goes wrong in prod, are you going to point your copy of 'some JS code that someone put on the Internet' and still have a job?

https://docs.aws.amazon.com/secretsmanager/latest/userguide/...

https://www.hashicorp.com/trust/compliance/vault

Yeah, many probably wouldn't get fired for that, but small consolation for a breach.

The very fact it was audited massively reduces the chances it’ll be breached compared to a random JS file that hasn’t been seriously audited. A “please read and tell me the problems” is NOT a security audit.

I was only addressing this part: "and still have a job".

Clearly well audited code is likely safer.

I just don't think that screwing that up will definitely lead to most being fired.

You can’t separate “auditing can reduce chance of breaches” from “using unaudited security critical software when certified alternatives exist can be gross negligence.”

The criticism in that issue is pretty bad, I agree. But the crypto in secrets.js is all kinds of bad:

The use case is sometime calling this tool to decrypt data received over an unauthenticated channel [0], and the author doesn’t seem to get that. The private key will be used differently depending on whether the untrusted ciphertext starts with '$'. This isn’t quite JWT’s alg none issue, but still: never let a message tell you how to authenticate it or decrypt it. That’s the key’s job.

This whole mess does not authenticate. It should. Depending on the use case, this could be catastrophic. And the padding oracle attack may well be real if an attacker can convince the user to try to decrypt a few different messages.

Also, for Pete’s sake, it’s 2025. Use libsodium. Or at least use a KEM and an AEAD.

Even the blog post doesn’t really explain any of the real issues.

[0] One might credibly expect the public key to be sent with some external authentication. It does not follow that the ciphertext sent back is authenticated.

But having bad crypto doesn't mean you have to be aggressive... in fact if the critic's goal is to actually improve the situation (and not just vent or demonstrate their superiority) then being polite and actually answering the questions might go a long way further to remedy it.

You’re right. The problem is that after repeating the same thing hundreds of times to different developers you can develop a bit of an anger toward the situation, as you see the same mistakes play out over and over.

I’m not defending it, but I can understand where it comes from.

What's the point of filing the issue if you're already fed up giving adequate answers then? For a random open source project I'd definitely expect people who file issues to help solve them collaboratively.

That was my question too, honestly.

Why is soatok filing the issue if they are unwilling to explain it in a way that doesn't drive traffic to their blog?

Filing an issue and subsequently refusing to elaborate is also awfully suspect.

I don’t disagree with you. The idea is that there is a lot of information online about how to do it right. I would have stayed to explain, because it’s confusing af, but I think that’s what happened. I don’t think soatok is intentionally self-serving, but I don’t know them personally.

Only tangential to this post, but if you need a way to share secrets with your teams (or whoever), Hashicorp Vault is pretty decent. They don't even need login access. Instead of sharing secret values directly, you wrap the secret which generates a one-time unwrapping token with a TTL. You share this unwrapping token over a traditional communication method like Slack or e-mail or whatever, and the recipient uses Vault's unwrap tool to retrieve the secret. Since the unwrapping token is one time use, you can easily detect if someone intercepted the token and unwrapped the secret (by virtue of the unwrapping token no longer working). This hint tells you the secret was compromised and needs to be rotated (you just need to follow-up with the person to confirm they were able to retrieve the secret). And since you can set a TTL, you can place an expectation on the recipient too -- for example, that you expect them to retrieve the secret within 10 minutes or else the unwrapping token expires.

All of this has the added benefit that you're not sharing ciphertext over open channels (which could be intercepted and stored for future decryption by adversaries).

I like this. Can you add yubikey as another factor?

Where I work we never need this though, we have a jwt server that can serve a time limited token for work account that various systems can accept.

Glad to hear this. I’m planning to use Vault in a new project that has sensitive security concerns. I liked Hashicorp’s concept of “encryption as a service” as a way of protecting engineering teams from cryptographic footguns.

I'm not sure the way to get developers to stop writing their own crypto is to turn it into a delicious forbidden fruit edible only by the most virtuous. APIs sometimes have a "Veblen good" character to them: the more difficult, the more use they attract, because people come to see use of the hard API as a demonstration of skill.

The right way to stop people writing their own cryptography isn't to admonish them or talk up the fiendish difficulty of the field. The right way to make it boring, not worth the effort one would spend to reinvent it.

I think one of the reasons developers roll their “own” crypto code is because there doesn’t really seem like a simple “do it this way” way to do it. OpenSSL has hundreds of ways of doing encryption. So it almost has pitfalls by default, because of the complexity of the choices.

OpenSSL is not misuse resistant, nor is it opinionated.

Opinionated means the ciphers available are the best of the bunch. Misuse resistance means the API validates parameters and reduces the footguns.

If you're doing a closed ecosystem, you'll want both. Use something like libsodium, or it's higher level wrappers. For this the team should have decent understanding of the primitives of that high level library to e.g. know that insecurely generated keys are still accepted so you'll have to use a CSPRNG.

If you're having to interact with other systems that only support e.g. TLS, you'll want to have the company hire a professional cryptographer with focus on applied cryptography to build the protocol from lower level primitives. Other programmers should refuse to do this work, like non-structural-engineers refuse to do structural engineering because they know they'll be held responsible for doing work they're not qualified.

In the end the author expressed his frustration at the lack of input from security professionals, but the words he used were perhaps a little arrogant:

He said "to routinely make preventable mistakes because people with my exact skillset haven’t yet delivered easy-to-use, hard-to-misuse tooling".

I would suggest rephrasing this as a way to make it clear what skills he is referring to, something like "people who are senior security experts". Otherwise it might sound like he is implying that he is the only one who should audit everything, because who else would have the exact same experience that he has had all his life?

> I would suggest rephrasing this as a way to make it clear what skills he is referring to, something like "people who are senior security experts".

Well, it needs to say exactly what it says, not a vague category like "senior security experts".

There are countless developers and security nerds who run circles around me. They can do everything I can do, and more.

A lot of the trappings that cause developers to make preventable mistakes is because my betters and I haven't delivered easy-to-use tools that solve their use cases perfectly, and I feel personally responsible for not being able to help more.

Not sure what's arrogant about that.

Well, just thinking about the meaning of the word "exact" can be literal, the level of frustration suggests the emphasis is the highest, it can make the reader understand the meaning of "exact" is a literal irreducible exact, implying no one other than you have the literal exact same experience.

I'd argue there are easy-to-use tools for this use case, frankly. Buy your employees corporate Yubikeys, issue them over certified mail, and just use gpg in some authenticated mode. That may be too complicated for Grandma Marie and Uncle Paul, but is it seriously too complicated for someone trying to run a software business? How do you ever expect to understand the laws you have to comply with to run a business if that's your attitude?

If that's still too complicated, send each other API keys over Proton Mail. Unless you're an enemy of the Swiss government, I can't think of a reason not to trust them that isn't into serious crackpot territory. If you're actually being targeted by Mossad or the NSA, they can intercept your certified mail anyway. OpenAI would probably cooperate with them besides.

I have empathy for people who end up stuck in the in-between areas, where out-of-the-box building blocks solve part of their problem, but how to glue them together can still get tricky.

For one example, you've got a shared secret. Can you use it as a secret key? Do you have to feed it through a KDF first? Can you use the same key for both encryption and signing? Do you have to derive separate keys?

I'm in that boat. I'm watching all of Christoph Paar's cryptography lecture series on YouTube -- it was recorded in 2010, so I do wonder if it's missing any new state of the art / best practises.

I'm like 18 lectures in, two out of three semesters. And I still feel like I have only the vaguest ideas what the primitives are, how they work, what they're for, and their weaknesses. I'm having to follow all the mathematics as someone not mathematically inclined (Prof Paar did do a good job of making the mathematics fairly accessible though).

All of this so I can have a bit more confidence in proposing E2E for a project at some point in future (before somebody asks us to, too late).

And my use-case makes it difficult to follow the most trodden paths so I can't just plug in a handshake protocol and MACs and elliptic curves or "just use PGP" or whatever.

As a software dev, I have all these boxes I could use, that come with so many caveats "if you do this, but don't do this, no do that, don't do that"... It's very tricky trying to work out how to glue the pieces together without already being in the field of crypto. Feels like I'll always be missing some crucial piece of information I'd get if I pored over hundreds of textbooks and papers but I don't have the resources to do so!

I'd love if someone did like, a plain English recipe book for cryptography! Give the mathematical proof of stuff, but also explain the strengths/weaknesses/possible attacks to laypeople without the prerequisite that you need to understand ring modulus or Galois fields or whatever first. Or, like, flowcharts to follow!

>so I do wonder if it's missing any new state of the art / best practises.

https://nostarch.com/serious-cryptography-2nd-edition should have the latest info, it's approachable and goes into pitfalls. https://www.manning.com/books/real-world-cryptography is another.

>As a software dev, I have all these boxes I could use, that come with so many caveats "if you do this, but don't do this, no do that, don't do that"... It's very tricky trying to work out how to glue the pieces together without already being in the field of crypto

Until you know more, strongly consider suggesting the company just hires someone who knows that. Just because you're available to do it, doesn't mean you should just yet.

Thanks, I'd not found these yet! Very helpful :)

> Until you know more, strongly consider suggesting the company just hires someone who knows that. Just because you're available to do it, doesn't mean you should just yet.

This is a fair point. We'd always find it difficult to hire someone who was 100% specialising in software security / crypto etc, but a software eng who has some experience would probably be palatable... But funding for new hires could be a couple of years out. That, or we find a way to turn it into a research proposal we can sic a PhD on.

Still, I think it benefits us to have a strong baseline knowledge of crypto systems as a team, "bus factor" and all that. Maybe one day we have a colleague that can teach us that, but until then we may as well crack on with self-teaching :-)

My question is, why does the library even support AES-CBC? I know that it's OpenSSL here but why can't we have an additional library then that ships only the recommended encryption schemes, safe curves, etc. and deprecates obsolete practices in an understandable process? Something that is aimed at developers and also comes with good documentation on why it supports what it does.

Legacy. You rarely only touch new stuff or secure stuff. And instead end up at least with one scenario that requires you do something with old stuff. And this is not just crypto.

I get that. But then, keeping that legacy stuff running is just as problematic as rolling your own crypto. We can leave OpenSSL as it is but it shouldn't be the popular recommendation for developers.

They should have another library which, like I said, actively deprecates obsolete and insecure practices but in a way that makes the update process digestible for people depending on it.

> I get that. But then, keeping that legacy stuff running is just as problematic as rolling your own crypto.

Both problems are from the lack of knowledge, but latter one is orders of magnitude harder to fix.

>We can leave OpenSSL as it is but it shouldn't be the popular recommendation for developers.

OpenSSL isn't a recommendation for developers. For TLS, LibreSSL and BoringSSL are. For other stuff, libsodium is.

The only reason I've picked OpenSSL, is it has a higher level library (pyca/cryptography) that gives bindings to X448.

Because such library is pretty useless in the real world where your system needs to interact with other systems using "legacy" encryption schemes.

Usability and practicality is critical for successful security approach.

My optimistic opinion is that since our information is being leaked left and right, it doesn't matter whether we roll the algo by ourselves or not. Might as well do it for practice...

Quote from the Honorable MI5 head:

> I mean, with Burgess and Maclean and Philby and Blake and Fuchs and the Krogers... one more didn't really make much more difference.

Ah, yes, common people and cryptography. I've recently been in a meeting with industrial people, the topic was securing industrial communication in production lines and such stuff: "Someone told us zero trust means we can establish secure communications between random devices without any root of trust". I think they plan to eventually standardize something around that. I'm looking forward to either be pleasantly surprised, or do a side quest to prevent them from making fools of themselves.

Aside: I appreciated this article and didn’t find it overly arrogant as others have suggested. It sounds like a security expert venting a bit after being exasperated. His gripes are legitimate, and his tone wasn’t out of line with the frustration I’d feel in the same situation. My 2cents.

The reason why cryptography is so bad.

Is this elitist culture in cryptography.

Alot of algorithms are described incomprehensively.

Let me give you an example.

You might get a well documented specification for implementation of ECDSA.

But it will lack something very 2 important concepts.

1. You should only use it with "safe curves"

2. It's does provide you with a list of "Safe curves"

Same goes for Shamir Secret Sharing. You can know how to implement the algorithm.

But there is so much extra "insider knowledge" or "tribal knowledge" that isn't obvious to non-cryptographers, that you need, for it to be secure.

Such knowledge is often (if not always) not documented. It's in obscure forums, papers and blog posts like this one.

There is no comprehensive encyclopedia for cryptography algorithms.

Developers write bad encryption code because cryptographers have bad documentation.

I would blame the cryptographers, not the developers.

What a coincidence! I was just browsing the Shamir's Secret Sharing Wikipedia page 30 seconds ago. There is a python implementation on it and I was worried the same exact thing as you before opening HN. So maybe we could start with that one. Is that code implementation sufficiently secure and well documented?

https://en.wikipedia.org/wiki/Shamir's_secret_sharing?wprov=...

https://zkdocs.com has a whole chapter on Shamir's Secret Sharing.

(Ironically, this stuff is documented, cryptographers just aren't good at marketing.)

https://www.zkdocs.com/docs/zkdocs/protocol-primitives/shami...

I think this chapter just proves GP's point, rather than refuting it. It's much cleaner than the Wikipedia article and it mentions the zero share problem, but it doesn't mention any other concern like constant time implementations, cache side channel attacks or the leading zero coefficient that I pointed to at the other post.

It's a good guide to learn about Secret Sharing, but it doesn't give you even 1% of what you need to know implement it safely by yourself.

> or the leading zero coefficient that I pointed to at the other post.

That isn't actually a problem, although I've seen a lot of people that think it is.

The problem is framed as "if you have a leading zero coefficient, it's equivalent to a threshold of t-1", but you'd need a zero leading coefficient for every polynomial.

With a 32 byte secret and GF(2^8), you expect at least 1 leading zero coefficient in 11% of random secrets, but a threshold reduction from t to t-1 only occurs with 2^-256 probability (that is, every leading coefficient has to be 0).

You might think you can detect this condition, but SSS is kind of like a one-time pad if you don't have sufficient shares.

> it doesn't mention any other concern like constant time implementations, cache side channel attacks

These are table stakes for secure cryptography. ZKDocs is a guide to algorithms, not implementations.

No, it is not.

The arithmetic used is not constant time, meaning the actual computational steps involved leak information about the secret, were either the recombination of the shares or the initial splitting were observed via side channels.

The arithmetic does not guard against party identifiers being zero or overflowing to zero, although it is not likely to occur when used this way.

Case in point: http://sbudella.altervista.org/blog/20230330-shamir-timing.h...

I think that specific attack is pretty hard to pull off pratice (in normal scenarios, Vault secret unseals do not happen very often). But it can be a big problem if you were using a scheme where Shamir Secret sharing is used frequently (e.g. it can be triggered by a request sent by the attacker) and (I believe) with different parts of the secret every time.

It's probably safer to use an audited library, but here's a library library that has been audited by two firms, and it's still using lookup tables:

https://github.com/privy-io/shamir-secret-sharing/blob/main/...

The strangest thing is that the audit report by Cure53 mentions this issue, and says it was fixed, but it doesn't seem fixed (at least not in the way that I would expect and the way that HashiCorp fixed it, which is simply removing the tables and using constant-time math). The library maintainers seem to be very proactive and diligent about fixing other issues[1], so it really is strange.

[1] https://privy.io/blog/zero-leading-coefficients-cryptography

I don't understand what "blaming" the cryptographers gets you. Your system is either secure or it isn't. If you ship something insecure, that's on you.

There's definitely degrees of security that I'm having to constantly remind our security folks of when they aren't forced to deal with the costs of having expensive threat models.

If you're having to constantly remind your security engineers about "degrees of security," they may be trying to tell you that your threat model is unsustainably/nonsensically/dangerously lax.

The part I've been struggling with is getting them to put any threat model in writing that's not quote, "everything".

"Blame" isn't the most productive way to frame the conversation, but as GP mentioned, there aren't many resources for developers to learn about cryptography. We're in desperate need of comprehensive resources that can (at a minimum) help us answer:

1. Has this problem been solved using a proven cryptographic protocol? Which guarantees does this protocol offer? What are all the important considerations?

2. What's the next closest thing?

3. Can we safely modify the protocol in any way to accommodate our use case, and what are some common pitfalls?

Some people will ignore all that advice and do dumb things anyway, but if we want systems to be more secure, it's on cryptographers to make cryptography more approachable and on developers to actually listen when they do.

It's not insider knowledge, it's just the collective experience people have built up over time after discovering issues with classes of ciphers. It would be nice to have a standard checklist that everyone could refer to (think OWASP Top Ten) while reviewing a cryptosystem.

The checklist is really simple: did you use a verified product or did you try to roll your own and fail?

Cryptography isn't like that, it doesn't boil down to 10 simple steps. If you want ten, it's happening at higher level

1. Never invent your own primitives. Only professional cryptographers do that in algorithm competitions like AES, SHA-3, or PHC.

2. For interoperability, use BoringSSL, LibreSSL etc. Hire developer who knows how to do this. Or, hire a professional applied cryptographer to build TLS from low level primitives if you have a reason it's needed.

3. For closed ecosystems, use opinionated, misuse-resistant, high level libraries like libsodium, or libraries binding to it https://libsodium.gitbook.io/doc/bindings_for_other_language...

4. Use kernel CSPRNG to generate all secrets.

5. Have the implementation audited by professionals. E.g. Kudelski Security or NCC Group.

6. Have a bug bounty program that compensates well, and that allows disclosure of the attack.

7. If you're in a market where the competition has open source clients, you should have an open source client too, so that anyone can verify your cryptography is correctly implemented.

8. Keep anything that's WIP, under "research prototype, do not use in production" banner. That's not a cone of shame. If Dan Boneh can do it (https://crypto.stanford.edu/balloon/), so can you.

9. Realize that 90% of the work in cryptography is key management. Use OS keyring when possible.

10. Deploy end-to-end encryption everywhere you can. Data is a toxic asset. https://www.schneier.com/blog/archives/2016/03/data_is_a_tox...

So yeah, sorry, no "Use XChaCha20-Poly1305 or AES-GCM only", or "Use X25519/ed25519" or "remember to hash the X25519 shared secret". That's too in-depth for a ten point list. At that level, it's more of a top-ten book list.

>Is this elitist culture in cryptography.

It's elitist in the same way quantum physics, string theory and molecular biology are. A hard science with lot's of moving parts not everyone can manage.

>But there is so much extra "insider knowledge" or "tribal knowledge" that isn't obvious to non-cryptographers, that you need, for it to be secure.

Misuse-resistant cryptography is an open research problem. Claiming it's "tribal knowledge" makes it seem nefarious. Academics are not always good teachers, and collecting all the lessons from all papers you read into a book, is time away from doing more research that advances your career.

>Such knowledge is often (if not always) not documented. It's in obscure forums, papers and blog posts like this one.

You can pay for the papers, and you can pay for the school to learn to understand those 'obscure' papers. Or you can hire someone who has done that. If you're a developer expected to implement algorithms of theoretical physics, you're not complaining about the papers being obscure, you're hiring a physicist with double-major in computer science.

There's way too much frustration towards cryptographers, and way too little frustration towards bad hiring practices, and incorrectly assigned work.

I did not learn a single thing from this article. I did learn quite a bit from the programmer's stuff and situation.

We were doing an integration with a partner for our customers, and the contact I was working with insisted on using some OpenSSL primitives that were exposed in PHP:

(a) they reversed the public + private parts of the key, and were upset when I communicated the public part of the key in cleartext

(b) they speced that the string being encrypted could not exceed 8 bytes ......

I tried so very hard and very patiently to explain to them what they were doing wrong, but they confidently insisted on their implementation. To deter fellow devs from trying this, I left loud comments in our code:

> So these guys are totally using RSA Crypto wrong. Though it's a PK Crypto system, they insist on using it backwards, and using signatures to send us cateencrypted values, and we send encrypted values back to them. It's dumb. I suspect someone read through the PHP openssl function list, spotted RSA_encrypt_private and RSA_decrypt_public and decided to get overly clever.

> This consumes a public key, and uses it to 'decrypt' a signature to recover it's original value.

> To further deter use, I will not add additional documentation here. Please read and understand the source if you think you need to use this.

Reminds me of a vendor providing an XML-RPC API with their enterprise product. The customer had a requirement that all PII information be encrypted in transit, and this was used used to send personal information about minors.

I expected them to simply turn on HTTPS like normal people.

Instead after months of effort they came back with XML Encryption. No, not the standardised one, they cooked up the their own bespoke monstrosity with hard-coded RSA keys with both the public and private parts published in their online documentation. The whole thing was base-64 encrypted XML inside more XML.

I flat rejected it, but was overruled because nobody involved had the slightest clue what proper encryption is about. It looked complicated and thorough to a lay person, so it was accepted despite my loud objections.

This is how thing happen in the real world outside of Silicon Valley.

The was at least also TLS on top of that, yes?

Of course not! Plain-text HTTP only, because “that’s simpler to debug”.

> The whole thing was base-64 encrypted XML

Surely you mean "base64 encoded"

Yes, I meant "base-64 encoded encrypted XML", but I accidentally a word.

[flagged]

> they reversed the public + private parts of the key

Is there a reason this is actually a problem? I always thought the public/private key was arbitrary and it was easier and less error prone to give people one key called public and one called private than hand them two keys and say "ok keep one of these a secret".

Don't get me wrong, not defending anyone here, just curious is there's more to it I don't know.

I'd assume that the value intended for public exponent is used as private key exponent. Typically, public key exponent is very small compared to private key exponent. This means that the private key exponent is very small in their scheme, so attacks such as Wiener's attack[0] can be used to break the encryption.

Also, I'd like to add that public exponent is usually fixed to some well-known constant such as 65537, so the attacker might just try brute-forcing when she knows the details of the scheme.

[0]: https://en.wikipedia.org/wiki/Wiener%27s_attack

That's not generally true: typically you can compute the public key from the private key, but not vice versa (i.e. crypto_scalarmult_base() in libsodium).

In RSA, that isn't possible (without assumptions), so a private key file also contains the public key data.

Honestly, I don't know. And as a non-expert on all things crypto, I delegate my trust into the labels that the expert do assign things. There may be a weakness in the algorithm if the keys are reversed, that is not immediately obvious. If the thing we're protecting is worth protecting, it's worth doing it right...

> To err is to be human, but to routinely make preventable mistakes because people with my exact skillset haven’t yet delivered easy-to-use, hard-to-misuse tooling in the programming languages actual developers use–meeting them where they are, as it were?

This paragraph shifts the blame around to the implentors of cryptographic libraries.

Some cryptographic assumptions can be enforced by a type system and runtime checks. But we probably still need to have good reading resources to learn about how to properly use common cryptographic algorithms so that developers can recognize when there is something awry with a cryptographic library. And these reading resources need to be well-known in the community to have an impact. Does anyone here know of some good books or good websites that explain the things that developers need to know?

> This paragraph shifts the blame around to the implentors of cryptographic libraries.

I think a subtlety here didn't scan for most HN users: I'm blaming myself when I write that.

Building the tools that solve your exact needs is my hobby horse. Providing developer tools that are easy to use and hard to misuse is a significant time investment of mine. For more on that, read any of my blog posts or GitHub repositories over the years.

It's extremely frustrating to me that, despite all of the effort I've put into this problem (both as my blogging persona and professionally), people still think "I'll try to encrypt secrets using Node's crypto module" in 2025, and end up with less secure implementations.

(If you want to go down this route of inventing it yourself, rather than using an off-the-shelf solution, start with hpke-js and forget RSA exists entirely.)

> I'm blaming myself when I write that.

The way I understand it: You are blaming yourself (and people in a similar position) because you know better and you have the ability to create better libraries.

But as someone who does a lot of software engineering, I also recognise that writing reliable software is time-intensive. And writing library code that doesn't have any cryptographic flaws is probably extra time-intensive, because there is no crypto-aware type checker or linter watching over your shoulder, because afaik these tools don't exist. Good cryptographic software is not written in a day and I know that this is very frustrating. I've got a lot of half-finished side-projects laying around...

But my original sub-point still stands: We need better well-known learning resources on how to use cryptographic algorithms correctly, so that we as developers can recognize misuse of/by libraries. And I'm happy to hear any suggestions for books and websites.

s/Encryption/Security

Roll-your-own cryptography is definitely the worst of it, but even developers that use strong crypto libraries end up misusing them quite often.

I have worked on a lot of custom made web software, and I could count the number of in-house authentication or authorization systems that didn't have glaring issues on one hand.

After nearly 12 years of working in this field, I would consider myself extremely knowledgeable in web security and I still don't think I'd be comfortable writing a lot of these types of systems for production use without an outside analyst to double check my work. Unless you're a domain expert in security, you probably shouldn't either.

I think what the author implies is that people should use trusted and vetted crypto libraries, and rollout their own encryption primitives based on some documentation they read from who knows where.

Look at OpenSSL for example, even though the people who work on the project know a lot about crypto, they still make big mistakes. Now imagine the average developer trying to build crypto from scratch.

The reason overconfident developers roll their own crypto is because the ‘cryptography experts’ are such asshats about it. Who wants to deal with someone that writes this?

Hell, even identifying which ‘cryptography expert’ actually knows what they’re talking about and which ones are windbags is often more trouble than it’s worth.

I don't love the post either but getting angry about it doesn't make it less right.

I’m not exactly angry at it. Just pointing out the flaw in its argument.

It’s never worked to yell at my junior developers and tell them that ‘hell is them doing any kind of development’.

It works better when I just show them what they should do instead.

>Hell, even identifying which ‘cryptography expert’ actually knows what they’re talking about and which ones are windbags is often more trouble than it’s worth.

If they look like they spent too much time in the sun you can't trust them. This criteria has yet to fail me.

The more nuanced message is maybe don't write crypto code without having spent the right amount of time to understand the challenges. If you don't know about timing attacks, you aren't familiar with the many historical mistakes that are well known, you don't know any theory around ciphers or hashes, then you probably shouldn't be writing crypto code that is used to secure anything important. There are a lot of ways to get up to speed (Coursera's cryptography courses from Stanford, books etc.) to at least wrap your head around why this is a problem. Even the experts go through peer review cycles because they know how easy it is to make mistakes.

In other news, don't print yourself a motorcycle helmet on your 3d printer, don't make your own car tires, don't make your own break pads. I mean if you really know what you're doing maybe.

EDIT: I have some personal experience with screwing up some crypto code. I won't get into all the details but I was overconfident. And this was after taking graduate level cryptography courses and having spent considerable time keeping up to date. I didn't write anything from scratch but I used existing algorithms/implementations in a problematic way that left some vulnerability. This was a type of system where just using something off the shelf was not an option.

> There are a lot of ways to get up to speed

None condoned by your employer however. Those tickets need finished yesterday. Security is just a checkmark.

I just skimmed the article, but I generally agree. I mean, I get it - I think it's actually pretty easy to make an argument "Secure crypto code is exceedingly difficult to get right, even for experts. And unlike lots of other areas of code, there is no 'halfway broken' - crackable crypto is 'game over' in a lot of situations, which is why it's so dangerous to roll your own."

But there's a huge gulf between that message and "Hell Is Overconfident Developers..." And importantly, I don't think that "overconfident developers" is the biggest problem. If anything, even when I very much don't want to "roll my own" crypto, I've found it difficult to be sure I'm using crypto libraries correctly and securely. Sometimes libraries are too low-level with non-obvious footguns, or they're too "black box" restrictive. I think a great example is the NodeJS crypto library. I think it's a great library, but using it correctly requires people to know and understand a fair amount about crypto. Whenever I use node crypto I pretty much always end up writing some utility methods around it (e.g. "encrypt this string using a key to get cipherText" and "decrypt this ciphertext using this key"). I think it would be better if those standard utility wrappers (that do things like use recommended default encryption algorithms/strengths, etc.) were also included directly in the crypto library.

I think most developers are actually scared of rolling their own crypto, but cryptography experts don't make it easy to give them a plug-and-play solution.

Based comment, updooted. Also, I'm pretty sure that if nobody ever "rolled their own crypto" we would never have new innovations in cryptography. By OPs argument everyone in the blockchain industry is in a state of error since using applied cryptography to design new protocols is certainly novel. But I would say that these protocols are what made me interested in applied cryptography in the first place. They are astonishingly clever, novel, and show how beautiful cryptography can be. If everyone was too scared to work with cryptographic constructs than none of the innovation that took place around smart contracts would be possible.

I see no problem with it if your work gets peer reviewed. The very last point they made though I kind of agree with. If you end up making a new algorithm from scratch that is dangerously novel and potential crack pot territory. I did like how they went to that trouble to make the hierarchy of what is meant by "rolling your own crypto."

I kept waiting for actionable advice, like maybe a site somewhere documenting the current best practices on common tasks, say securely implementing the complete lifecycle of encrypted sessions. Got to the end and it seems the advice, if there is one, is to pay some cryptography expert to review my system. And I have no idea if the author is a cryptography expert or an overconfident expert-wanna-be rolling their own crypto advice. So yeah, I agree.

Can HN give some actionable advice on dos, not don’ts (I already know all the don’t like don’t use OpenSSL primitives, don’t use AES-CBC, blah blah)?

plenty of "cryptography experts" that you blindingly trust because "don't roll your own crypto" make giant, fatal, mistakes too. i love how the "crypto is so hard, even experts get it wrong" argument both supports the elitist POV and is apologist for when "even" the experts get it wrong.

crypto is like many other things in which if you make ONE TINY mistake you are doomed. point taken. do your due diligence. but also don't trust randos. or the NSA and probably NIST. therefore most so-called experts.

if hell is "overconfident" developers, then what is "overconfident experts". or "overconfident furry apologists for crypto experts"? also hell.

crypto is hard. we get it. you're also no better than anyone else at it.

It really needs to be taught in comp sci classes at this point. Have an entire project dedicated to why rolling your own crypto is at best a waste of time. Students will listen if they need to pass a test on it.

Giving people more introduction to crypto is a good way to get more handwritten crypto. It's part of the fun family of purely mathematical algorithms, alongside hashes, RNGs, and checksums. People make essentially the same mistakes in all of these.

Do you not want to train some of those students to write the improved cryptography of the future?

I do agree having a project where you demonstrate problems they introduced (by exploiting them directly, or have another student team be the exploiters) would highlight the risks, but school is about teaching, not about scaring people off.

That’s what the actual crypto classes are for. If people are intrigued, those are fantastic elective courses.

Nah, hell is being stuck writing Kubernetes configs.

Yeah, devs aren't meant to roll their own crypto (tm) but most open source libraries for doing crypto are dangerous (and lack clear guide lines for using them.) So then, what do you expect people to do? OpenSSL is hugely over complicated and hard to use. Having it as a dep almost certainly means breaking your software at some point, too. Libsodium is damn good but you still have to go out of your way to work with it if you're not using its baby-tier box design.

There are safe / standard ways to use certain primitives. But then there's no guarantee that such algorithms will be made available in the libraries that you're trying to use. E.g. lets say you want to use RSA (for x509 certs or something like that.) You'll get access to RSA but no algorithm to handle padding and no mention of it. So the dev uses RSA as-is and then gets a lecture for padding.

Point is: I don't see many "cryptography experts" putting effort into improving developer documentation resources for libraries. But plenty of them ranting and acting butthurt that developers aren't educated enough to use the primitives. The same crypto experts will also release these libraries and then cry that people use them. Lol, lmao even?

> I don't see many "cryptography experts" putting effort into improving developer documentation resources for libraries.

This isn't a "documentation resources for libraries" problem. It's a what libraries get bundled into the "crypto" module for your programming language problem, and that's largely a political decision rather than a technical one.

For example: Node's crypto module will always be a thin wrapper around OpenSSL, due to backwards compatibility. I cannot change that reality. There will always be a way to shoot yourself in the foot with Node.js's crypto module, and that is what most developers will reach for.

Even if I introduce new APIs for Node crypto and update the official docs to recommend the newer, safer ways of doing things, I won't be able to update the long tail of Stack Overflow answers with 10+ years of incumbency and hundreds of upvotes. This was tried years ago in the PHP community, and you still find bad PHP code everywhere.

Updating the developer documentation only serves to blame the poor user when they fuck up, so you can dismiss their pain with "RTFM".

> But plenty of them ranting and acting butthurt that developers aren't educated enough to use the primitives. The same crypto experts will also release these libraries and then cry that people use them.

I have never contributed a line of code to OpenSSL. You do not hear Eric A Young "ranting and acting butthurt". What empirical evidence do you have for this assertion?

Sure, even RSA can be used safely. This site's root certificate is still 4096-bit RSA, and that's probably OK (if we trust Let's Encrypt and our TLS implementations). If what your library exposed was a function called validate_x509_certificate_for_domain() that receives an X.509 certificate and a domain name and then validates it against your trusted root CAs then sure - you could write simple documentation on how to use it safely.

But if you want to use the general-purpose RSA functions, you're out of luck. The design of OpenSSL and Java's cryptography APIs is the stuff of nightmares, so let's take Go, which has a more modern RSA API[1]. Padding for encryption is not only mentioned, but mandatory (you either call rsa.EncryptPKCS1v15 or rsa.EncryptOAEP, you cannot use encrypt or decrypt RSA without padding if you really wanted to). The documentation would even nicely warn you not to use rsa.EncryptPKCS1v15 unless you need to maintain compatibility with a legacy protocol.

So can you just go ahead and use Go's "crypto/rsa" safely? No, of course not. RSA encryption (even with OAEP padding built-in) is such a low level operation that it would almost never be used alone. For instance, to encrypt a file with RSA, you will need to first encrypt an ephemeral random symmetric key, and then choose a symmetric algorithm and chaining mode. And in most cases, you also want to sign the envelope with your own public key.

So at the very least, if you want to build something like your own mini-PGP that is using legacy crap that can be somehow made safe just for the sake of sticking it to elitist cryptographers, you would need: A good RNG, RSA encryption (with OAEP), RSA signatures (with PSS), AES-CBC with PKCS#7 padding and SHA-256 (for the RSA signature). The documentation will have to mention all of that, and we still have only covered just a single use case: a toy mini-PGP. And let's not forget how to generate key. Go would always use e=65537, so we've got that covered, but you still need to know at least the recommended key size.

Unfortunately, unlike the baby-tier box design, our toy-tier mini-PGP doesn't have even perfect forward secrecy. So if we want to make it safe, and our requirements are to use prime number cryptography (because prime numbers are cool?) we'd have to add Diffie Hellman and that is it's own can of worms. But even with ecdh, we'd need to select a curve. Should the documentation should just tell the user to just select use ecdh.X25519, or should it go on explaining about Montgomery Ladder and Weierstrass scalar multiplication, invalid curve attacks and indistinguishability from uniform random strings[2]?

In the end of the day, we would end up with documentation the size of a couple of book chapters explaining how you could safely use a bunch of cryptographic primitives for one single thing (a toy mini-PGP). If we'd want to go further than that and explain how to safely implement something more complex, we'd end up with a couple of applied cryptography books. And if heavens forbid you'd want to implement a new protocol for secret messaging, we might have to include a few hundreds of research papers and perhaps a masters' degree program in cryptography.

And here we're back again to where we've been at the start. We've solved the issue by converting the random developer into an elitist cryptographer.

[1] https://pkg.go.dev/crypto/rsa#pkg-index

[2] https://safecurves.cr.yp.to/ind.html

Just add 2 to each char.

I... Don't really understand where to draw the line here. I'm not one to write AES implementations or something. But I did write a Noise implementation in C++ [1]. Does that count as "rolling my own crypto", even though I did my best to follow the Noise specification to the letter and used an audited cryptographic implementation? And the only reason I did that is because there wasn't such an implementation for C++. I won't deny that I'm not an expert of cryptography, but I think this article is a bit extreme, at least from my reading. We can't always use "off the shelf" implementations of everything, because maybe one doesn't exist, or one exists but it has no license (there are surprisingly a lot of these on GitHub), or it has a license incompatible with your projects license, and so on and so forth. But hey, what do I know? [1] https://github.com/ethindp/noise-cpp

I think I interpret the phrase "don't roll your own crypto" as expanding out to something like "don't roll your own crypto unless you're willing and able to prioritize solving any vulnerabilities that people report to you". Someone who is porting a well-regarded cryptography codebase by carefully going through all of the relevant code and matching all of its intentional behavior has a credible claim they're part of the exception.

People who have put in the time to learn the common failure modes of cryptography attempts and how to solve them could make the claim they're in the exception to some degree. Someone who can't imagine how they would begin evaluating a report about something like "ciphertext malleability" shouldn't.

I mean fair enough I suppose? I've always gone with the idea that "don't roll your own crypto" meant not implementing cryptographic building blocks. Which is reasonable and fair. I just take issue with it being something you'd apply to all crypto code.

> I just take issue with it being something you'd apply to all crypto code.

That this discussion is happening at all in response to a blog post that begins by linking to https://www.cryptofails.com/post/75204435608/write-crypto-co... is amusing and concerning in equal measure.

I don't care if you write crypto code. I never said I cared if you write crypto code. But don't:

1. Deploy your code

2. Publish your code in a package ecosystem

3. Encourage other people to use your code

...unless it's survived the gauntlet of peer review and has a good rationale for existing in the first place.

The reason I proposed an onion model for this discourse is that there isn't a binary switch where "if you do X you are rolling your own crypto but if you do Y you aren't rolling your own crypto". The deeper you slice, the more danger and responsibility you've incurred.

People who think "it's fine to build a custom protocol out of cryptography bricks because I'm not rolling my own bricks" are the highest risk group in software developers. Even moreso because (as tptacek constantly points out) most vulnerabilities occur at the joinery.

The insecurity is freedom. Think of that again when you need to break out of Big Tech's control.

What? Say more.

Big tech often uses cryptography to lock us out of our own hardware and into their DRM schemes, hiding what the software on the device is doing and often even hiding the binaries. Updates to the software become ways to more often protect the company's business models than to protect the user, and so "bugs become features" in this regime, leading to perverse incentivizes among everyone aware enough to understand this dynamic, with end users encouraging each other to NOT accept security updates in the HOPE that they will be broken one day in order to finally have a way to get root access on their own device. I recommend watching some of the many talks I have given on this subject or reading some of the many articles written by Cory Doctorow, if you somehow honestly don't know about this dynamic. When someone at Big Tech stupidly reimplements crypto... we cheer.

https://youtu.be/WyBgn06LNOY

I mean, that I get? I'm a professional connoisseur of cryptography vulnerabilities, so, for completely different reasons, I'm also happy to see things blow up. But like, the general sentiment of "it would be best if any given cryptosystem blew up", that sentiment I do not get. Platform security modules, sure (though: at that point you're conceding a lot to governments).

I didnt understand the point he was trying to make about trusting a public key from a remote server. At somepoint you need to trust some third party public key if you want to send them encrypted data and verifying ownership is kinda left to user. Hell even signal does that, who is really checking their contact security numbers to make sure the signal server didnt send you some bullshit...

> didnt understand the point he was trying to make about trusting a public key from a remote server.

What stops the server from swapping SyttenPK for NSA PK?

The operating word of the quote was "just".

> Hell even signal does that, who is really checking their contact security numbers to make sure the signal server didnt send you some bullshit...

Their latest code commits include key transparency, which is one good way to address this problem.

I do get that, but I am not so sure what the real solution is. The industry standard still is some form of "the server sends me a key and I trust it".

Key transparency is not really solving anything for the average person.

The problem isn't "the server sends me a public key".

The problem is there's no way to know if the server is lying.

Cool furry banner

… or any code?

Don't forget to rand(rand()).

I know people want to get paid but 90% of crypto is just about making sure the thing takes constant time, please stop acting like it is some elite knowledge

I have an entire crypto book on my shelf that only mentions constant time once in an offhand paragraph about timing attacks. They managed to write the rest of the book on the other 99% of crypto.

Reply on news.ycombinator.com