I do one controversial think and that's trim password imput (mainly because of trailing whitespace selection in some apps/oses). Other than that, if you can input it, you can use it... though now doing some unicode normalization for unity combos is probably a good idea prior to hashing.
If you choose four random words from a list of 2048 common words, and your attacker knows that's what you're doing, then your entropy is 4 * log_2(2048) = 44 bits. If the attacker didn't know your strategy and tried to brute force letter by letter it would be much higher - around 48log_2(26)=150 bits assuming around eight letters per word - but like you said, we should assume the attacker knows exactly what strategy we're using, so 44 bits is the better number to work with.
Since that's still more than 28 bits, a 'correct horse battery staple' type password is harder to crack then a short random string of mixed characters, even if the attacker knows exactly what your password generation strategy is.
I was a bit hasty of the entropy of the passphrase, my mistake. I still stand by that even if we choose from 2048 common words, generating a good passphrase (one that isn't a common sentence) is harder than we think.
Here, I just generated you a few passwords:
* hefty march attempt force bowel scuff
* between sepia book sweat lemma saint
* safe warn magical cask hefty wish
* alum glib puck adieu dour lazy
* telephone pine cavort good knee swank
* numeral plan jewel conch slate tube
* pastry piano sure proxy unit brew
* trig rise taint current sans gallop
Here is the same random numbers but encoded into ascii instead of words:
* 81Pk3t?Rq6S}
* ]CPcYrT^?iE3
* +qV`J9ZU&.,C
* `>sp=~V);3g>
* E&_ff7a|Z4B[
* ?OX~[J>0K'S*
These each have the exact same amount of entropy as the word-based ones.
I just watched a friend using some kind of long password and it took about a minute to get the password correctly entered. Easy to look at the keyboard also when he was typing and guess the words used.
Here's Merriam-Websters 3000 _core_ words [1]. Here's a list of 355k words [2].
That someone has selected 2048 words used to generate passphrases, doesn't make it easy to remember.
Everybody uses more than a few thousand words on at least one language. And selecting the least used ones will make your passprhases easier to memorize, because they have much more concrete meaning (by virtue of their rarity) than the most used words.
http://www.economist.com/blogs/johnson/2013/05/vocabulary-si...
The words don't have to be incredibly simple, either. Grabbing a random GfyCat image from the front page gives me "OrangeLankyBasilisk" which is easy to remember, none of those words are particularly foreign. But they're not in your list of most common words, nor are any of them in XKCD's simplewriter [1], which keeps track of the 1000 most common words.
[0] https://github.com/a-type/adjective-adjective-animal/tree/ma...
[1] https://xkcd.com/simplewriter/
Edit: The words need to be randomly chosen by a computer, so it doesn't matter what the most common words are. You could generate the password from a list of 2048 spanish words or japanese words or emojis and the entropy is the same as the previous scenario (assuming the attacker knows your dictionary of symbols just like they'd know your dictionary of English words). If you let a human choose, of course the smiley-face and poop emojis are going to be picked 50% of the time, but that's not the intention of the comic or passphrases.
If the human is picking anything than there WILL be a bias in selection. Effort should be made at minimizing that, but even with education this is a difficult task for any worker.
It got to the point where I actually took a classic literary work and made a 'password words' dictionary from it just so that I could have the computer generate possible new passwords. (there bias is mostly filtering out things that might be offensive... because someone can always take it the wrong way even if you explain in advance the theory of how the password was created by a computer).
> If you choose four random words from a list of 2048 common words, and your attacker knows that's what you're doing, then your entropy is 4 * log_2(2048) = 44 bits. If the attacker didn't know your strategy and tried to brute force letter by letter it would be much higher - around 48log_2(26)=150 bits assuming around eight letters per word - but like you said, we should assume the attacker knows exactly what strategy we're using, so 44 bits is the better number to work with.
> Since that's still more than 28 bits, a 'correct horse battery staple' type password is harder to crack then a short random string of mixed characters, even if the attacker knows exactly what your password generation strategy is.
Typo in the above but can't edit on my phone, the second sum should be 4 \* 8 \* log_2(26). Apologies.
A common sentence like "I drove to the mall yesterday" is not a good passphrase, but I'm certain that people who use "rocket" as a password would do something similar.
Choosing a sentence is a different strategy, which is less secure.
$ wget -O ⅓Mwords http://norvig.com/ngrams/count_1w.txt
$ for i in `seq 10`; awk '/^[a-z]{3,}/ { print $1 }' ⅓Mwords | head -n 2000 | shuf -n 5 | tr '\n' ' ' && echo
videos possible disease maintenance chair
teen documents than without son
research interface library largest drive
location ball beauty coming files
files middle fri meet air
guarantee samsung click super inn
legal previous rent resort use
reply thought better fresh phentermine
bad command once vehicle australian
fun random professor course sponsoredIt's a hard problem that is IMO best solved with hardware secure keeping of secrets and a rate limited pincode.
But you can't suggest that remembering 5 random words is harder than 20 random characters?
The goal is "simple" if possible, but "simpler" is still a lot better than "practically impossible".
We shouldn't have remember passwords at all IMO. It's creating entropy by remembering things, but the human brain is inheritly bad at remembering exact things. Things like a yubikey is a better idea, plug it in, enter your pincode, and use a key pair to authentication. All the user have to do is keep track of the physical thing and the pincode.
You use well-known the mnemonic trick demonstrated in the last panel of the comic.
A notable use case is choosing a master password for your password manager. And you'll want a longer phrase.
Here's a mnemonic generator I wrote in Rust https://github.com/leshow/rust_mnemonic for example
See https://en.wikipedia.org/wiki/Most_common_words_in_English for a claim. It is what teachers use as a guideline. It's the first 100 words they teach children to write.
> the idea is you use a mnemonic generator to pick the words.
I know you are supposed to use a generator to pick the words, that is how BIP39 for bitcoin works. But average Joe is not going to do that. He will select "I went to highschool in 1992". Authentication is a hard problem, and unless you force a reasonable scheme, it will be weak.
Edit: should have reloaded, said by enough people already :D