Basic Cryptography Concepts for Developers
paragonie.com
paragonie.com
"No one would ever expect the data to be base64-encoded twice! Mwa ha!"
There's less entropy per character if you know it will always be alphanumeric versus a string that of random bytes.
If you take 32 bytes from /dev/urandom and base64 encode it, it has the same entropy as the original binary string (unless you use it in a way that will result in it being truncated).
But the total entropy of the two strings is unchanged. One is just longer.
In short: You're correct, but with nuance that bears stating explicitly.
Now, ordinarily it would be bad enough that almost all of your clients are using the same encryption key, but it got much worse from here - turns out there was a bug in a proxy server they had written, so that every once in a while client B's traffic would be routed into client A's session, and hilarity would ensue; because the "keys" were the same, Client B's traffic looked like normal traffic from client A, so nobody could ever figure out why Client A would call us up and complain that he was getting fills for orders that he'd never sent! It took me forever to figure out WTF was going on, but one day I was watching a logging display and noticed garbled traffic showing up in Client A's logs, and it was garbled because it was a user who actually didn't auto-logon at 0930 in the morning and subsequently had a different encryption "key" resulting in his traffic not being decoded properly. From there, it became obvious what the problem was.
Edit: Yeah, yeah, I know - you're thinking "Wait, XOR? shared secret? But, but... how did they do key exchange?" Well, how the heck else would you send a client the secret key for the session - in plain text, when the client first logs on, of course!
"Password hashing algorithms" should probably be replaced with "key derivation functions" - which makes it clearer what they are, and makes further reading (by the reader) into KDFs easier.
- "KDF" says nothing about passwords (e.g. see HKDF) and,
- "PBKDF" (password-based KDF) says nothing about password verifier, such as for server storage (e.g. bcrypt doesn't produce a "key"), plus already taken by specific algorithms,
- "password hash" sounds good for both key derivation and verifier, but makes it easy to confuse with normal hashes.
(I've updated the title to read "Basic Cryptography for Developers" instead of "Cryptography for Developers" so as to not accidentally make someone think they know it all.)
It's a pity that very few customers are willing to pay for the effort to exorcising all (or at least most) of those demons in the systems I've seen deployed. The product differentiator (and hence the ROI) is almost always somewhere else. The science & engineering is vastly better than it was when I enterted the field, but it still has a long way to go.
For example: https://twitter.com/voodooKobra/status/651554578719227904
To try to combat this, I've been trying to help make basic security and cryptography knowledge accessible to web and mobile app developers, to hopefully result in an overall net gain for the security of many companies the world over.
Obviously I can't teach everyone everything there is to know about these areas. (For starters, I myself probably don't even know half of it. Lattice-based timing attacks? Not a clue!) But realistically if I can make a dent in the propensity for bad habits and worse design choices, it's something.
That said, my biggest lamentation is that most people don't seem to apply solid implementation principles to the systems they build. Witness vulnerabilities like heartbleed. It would have easily been avoided if the common buffer reuse guidelines in pretty much every implementation guide for security would have been used.
Some what conversely, I've proposed a model for classifying various forms of security vulnerabilities that might be easier for developers to conceptualize:
https://paragonie.com/blog/2015/08/gentle-introduction-appli...
The idea is treat it like a taxonomic model: You have general security mistakes (data-instruction confusion), which can be drilled down into vulnerability classes and then specific bugs that, either stand-alone or chained together, result in specific vulnerabilities in specific implementations.
Or maybe I'm way off base here. The feedback I've gotten has largely been positive, though.
The challenge is to get people with good critical thinking skills (like the kind you advocate, and embodied in proceses like the CC) and domain knowledge involved in building things. For example, I think I have a good ability to reason (I started my career in formal methods) but at present I know very little about how a browser actually treats it's inputs. So, it's quite difficult for me to reason about the application operating environment presented by a web browser running some javascript. I wouldn't trust myself to start coding a secure web site today. Hopefully the industry will foster both of these skills (a critical mindset & good domain knowledge) and put them to good use.
You are aware of Mitre's Common Weakness Enumeration, right?
This blog post is digestible in a few minutes. Crypto 101 can occupy a novice for a few hours. Given the hard choice of one of the two, we would encourage anyone interested to read Crypto 101 over our blog post, as they'll walk away with a much more detailed understanding .
PDF of the first and second editions available at [ https://www.cl.cam.ac.uk/~rja14/book.html ]
I've looked at the first chapter so far and I'm hooked.
I'm adding this to https://github.com/paragonie/awesome-appsec :D
/* Don't ever really do this */
It's not any more clear to see an example of how NOT to do something. And yes, despite all the comments, someone will copy and paste that into their program, remove the comments, and then never really properly update the code.
If nothing else, it aims to put to rest stupid phrases like "password encryption".
If people copy&paste despite all the warnings, they're beyond the help of any blog post.
EDIT: In the spirit of being more helpful, I've added links to more complete sample code (on StackOverflow) and made the warnings less obnoxious.
Also I'm not sure "password hashing" is any more descriptive than "password encryption."
I'll consider it.
> Also I'm not sure "password hashing" is any more descriptive than "password encryption."
It absolutely is. Encryption is a two-way transformation of data. It is, by design, reversible.
Hashing is one-way.
Password hashing is a special case of hashing where you want it to be computationally expensive (even with special hardware at your disposal) to attack, but still perform well enough to interact with.
Password encryption implies that a two-way transformation has taken place, and given the key, you should be able to reverse it. This is not within the scope of the requirements for secure password storage.
Idea was to obfuscate a strong password to make cracking more difficult. Plus, it was relatively easy to implement on arbitrary computers. These days I just memorize them or write them on paper.
The title is a reference to a meme ("You wouldn't download a car!") and also a reference to some absurdly bad cryptography, e.g. http://www.cryptofails.com/post/87697461507/46esab-high-qual...
Downloading a file from Server A and checking the hash delivered by Server A is security theater. In this case, only a digital signature (with a pre-established public key that you already trust) can really stop the server from being compromised (or malicious).
Downloading a file/torrent from Server B and verifying the hash delivered by Server A is a different situation entirely, and that boils down to a trust decision. Do you trust Server A to not be compromised? Do you trust them to not be malicious or in cahoots with Server B? If not, at the very least it's probably a larger attack service than the previous scenario. (Trusting the public key for the digital signature is also a trust decision. Only the details are different.)
Basically: If you're going to do anything at all, verifying hashes from the same source is a waste of CPU and human effort.
I hope that helps at all.
I also recall one or two stories where the binaries were infected but the hashes not updated - this was obviously caught pretty quickly and fixed.
However, I remember a time when Firefox served downloads directly from their mirrors. This case could be good for comparing hashes - but now it looks like they use Amazon's cloudfront.
But yes - for the average guy generating a hash for your releases (where your release and hash comes from the same server) doesn't provide any real benefit.