Randomly-generated passwords still have to be legal strings
blogs.msdn.microsoft.com
blogs.msdn.microsoft.com
This is like saying "Why isn't this random sequence of bytes a valid JPEG file?" If you want a random image, you generate the image randomly, then you encode it. Strings are no different — though the encoding is simpler.
The password bit is really just the hook, the main info in this article is the fact that these values are interpreted as UTF-16, and that UTF-16 has requirements beyond "sequence of 16-bit values."
Which is kind of obvious, but still not well known for many.
For UTF-32, only valid code points are allowed. Values above 0x10FFFF are not allowed, nor are values in the range used for UTF-16 surrogates.
UTF-8 is more complex. It's ASCII compatible, which means all values with the top bit set to 0 are allowed. Bytes with the top bit set to 1 must exist as part of a multi-byte sequence. The first byte in such a sequence must contain 2-4 leading 1s followed by a 0. Subsequent bytes must be of the form 10xxxxxx. The total length of the sequence is equal to the number of leading 1s in the first byte. Any deviation from this is invalid UTF-8. Further, any sequence must be minimal, meaning that the value it encodes can't be one that would fit into a smaller sequence.
Whether it may just so happen to represent real, correctly typed, English words, or be a valid ASCII string, or UTF-16 string, or whether it was typed it from your keyboard, or generated from a program, these should also be incidental.
232e2252b8d5a294dd8564020f0cf609f511b3d196ed83030ce9615c6737f578
That's readable, typable, copy-pastable. Not very memorable but 256 bits is rarely memorable.
It's still short enough to send around by normal messaging.
This is specifically about randomly generated passwords, this is not about dealing with human generated "must have capitals, not start or end with a number" etc rules.
232e 2252 b8d5 a294
dd85 6402 0f0c f609
f511 b3d1 96ed 8303
0ce9 615c 6737 f578
And before you Google and say "No! It changed to 127 in Windows XP!", remember:
- Somewhere on your network, NTLM is being used.
- Office 365 supports 16 max.
- Lots of random apps won't work.
There are only two possible hashes stored in the SAM database. You mention the 'shorter' one, which would be the 'LM hash' which is restricted to 14 character passwords. However, LM hash storage has been disabled by default for all windows system since Vista / Windows Server 2008 (and ability to not store them was available, but not disabled by default since at least Windows 2000). Even Samba has had the ability to not generate or store the LM hash since at least 2002 (earliest search reference I could find).
Using something like john [the ripper] with a decent dictionary and applying mangling rules will shred through the rest.
Not to mention, there's rainbow tables built and freely downloadable that contain every possible LM and NTLM hash (they're huge files, 30+ GB's, but if you need to crack in a hurry...).
Those would be the passwords that we're talking about here. Next time you run either, take a look at the count of uncracked passwords. There are always some, which should tell you something.
Also, I couldn't reply to you before so see my reply to parent WRT rainbow table size. They're not the panacea you're implying.
But they are - we have a hard-limited keyspace, with a fixed well-known and understood algorithm, which means, regardless of disk size, a rainbow table can (and has been) generated to cover the entire LM/NTLM space.
This place just has a few for starters[1]
I don't think this is true. Certainly true for the LM space of 7 characters, but the size of the NTML rainbow table for all passwords of length 1-9 with "only" mixed alphanumeric is 690GB with a key space of 13,759,005,997,841,642. This doesn't include the fact that a windows password can use any unicode character. So while there is could be some sort of collision with a shorter password, it's not really likely. Note that windows passwords can be up to 127 unicode characters, but that in win2k/2k3 the actual dialog box would only allow 32. So we're talking quite a bit more space to have each combination stored.
Additionally, unless I'm doing some bad math here, it's just not currently possible to have a rainbow table that includes all possible MD4 hash values. An MD4 hash is 16 bytes, which is 128 bits. In order to just list each hash would take 16 bytes * 2^128 = 5 * 10^39 bytes, which is something like 5 * 10^27 terabytes, 5 * 10^13 yottabytes, etc. This doesn't include any space to list whatever password caused that hash to be calculated.
And directly 'reversing' an MD4/NTLM hash still requires a preimage attack on MD4 and the current best known is still at a complexity of 2^121 [0], which would likely still be more feasible than having access to an MD4 rainbow table with all possible hash values.
[0] - https://who.rocq.inria.fr/Gaetan.Leurent/files/MD4_FSE08.pdf
What you're saying is akin to saying you can defeat AES by generating all possible keys and trying them. Possible? Totally. Realistic? No.
You have repeatedly now said that such a table exists. Source? How big was it, and at what work factor was it created? Keep in mind that the full space is all alt-codes and must cover passwords up to length 127. The site we have now both linked to does not have such a table.
Also, note that with the rainbow tables (for both NTLM and LM hashes), it will give you a password that hashes to the same exact value, but not necessarily the same exact password. This is kind of obvious once you understand that the NTLM (as compared to LM) hash is just md4(password) and is just 16 bytes long.
So that's valuable as it will let you log in to the machine, but since it's not actually giving you the actual password back, it's not helpful for logging in to other non-windows systems that may have the same password. Also, it requires administrator level access, a local privilege escalation exploit or a physical copy of the disk, to the SAM database, any of which likely mean that cracking the password is irrelevant.
I login to Office 365 with a password that has over 16 characters. I have similar length passwords in Windows networks that use NTLM. Clearly it works just fine.
Example:
Your password is 12 characters on a system that says make it 8 characters or longer. If you type only the first 8 characters of your password, it lets you in. As someone else said, "silent truncation".
All of this is anecdote from personal experience so take it with a grain of salt. I haven't researched it.
1 - if you have ntlm (non-v2) on your network, you're fucked anyway and this debate is academic. 2 - even if you do, having a password of 15 or more characters will cause your legacy hash to be set to "AAD3B435B51404EEAAD3B435B51404EE" and will protect you against having your hash cracked if it's stolen from the non-v2 service. 3 - the office 365 limits can be worked around with dirsync or federation (source: I've done both)(note: outlook does weird things with leading/trailing spaces) 4 - those apps should be kerberized or similar
In any case, the point is as a user, especially on a big network, you have no idea what your actual effective password strength is. Plenty of places do lots of dumb things with AD -- one example that comes to mine was an inter forest trust between two good sized domains where bad DNS implementation and... Misguided security interpretations basically made it impossible to use Kerberos.
$ dd if=/dev/random bs=1 count=32 | base64
ca2ktYYvppr1/HwwGYrg01udQX1cP4Ek5VmyzQydT+8=... when I just still wet behind the ears I was studying the codebase of my first job, and came across a quite extensive badwords.txt . On asking why we had a the dictionary of cussing in our project, I found out that the randomly generated passwords that we sent to our clients, apparently managed to get a customer quite angry when one of the words produced started with a slur, and this was a filter to prevent that from happening again
Generating random passwords isn't quite as straightforward as it would seem to be...
Isn't it easier to just generate words without vowels? This is the more common solution to that problem, IIRC.
Case in point: Not all scripts even have vowels.
> we sent to our clients
I'm making the assumption that these are one-time passwords for initial account setup. But no, no one scheme will work for everything, obviously. The hex version will fail for schemes which have an upper limit, or when there's a requirement for upper/lower/symbols. The base64 encoding will fail other validations due to the '/' character, or for particularly restrictive upper size limits.
I wish all of them would simply accept a string and put in an upper limit of 1k to limit DOS by password. Call it good.
Different systems have different characters that will be included/discluded from "twice click highlighting" (I have no idea what the term is called). Hex works on virtually all systems.
The article does mention it's not very space efficient.
No, it's not necessary, unbounded length is probably better. But it's pretty nice to have a lot more characters, and ought to be an easy win.
The goal should be long easy to type and remember passwords, adding an extra few digit is worth far more than forcing passwords to look like: 2K#@1j*
Compare say a 20 digit numeric password vs an 9 digit [0-9] [a-z] [A-Z] and special.
That is neither what the parent suggested nor really true. The space of, say, 8 character passwords A-Za-z0-9 that DO have at least one digit is very similar in size to the space of passwords maybe not containing digits. (The latter is 36^8 the former is 36^8-26^8, a reduction by less than 8% of the password space. The difference is even less important for longer passwords)
PS: You also forgot about the special character.
I don't understand your comments about adding digits and special characters at the end of the password. password + digit + special character is strictly harder to guess than just password (we've added two extra characters, after all). Do you mean that people routinely delete the last two letters of their alphabetic password and replace them with 1! ? I find that implausible. The key point is that if you don't require numbers and special characters then no one uses them at all so the possibility space becomes 26^8.
No, the same length just with more restrictions. Your suggesting you replace a min 8 digit password with a minimum 10 digit password but the gain is from adding length not restrictions.
Sadly plenty of passwords are fixed at 8 digits and they reduce the search space below that. So you go from 26^8 to 26^1 x 26^5 and 1! which effectively is just 26^6.
IMO it's reasonable to remove the top 100 or so passwords. But, after that your just reducing the search space.
100 passwords is nothing compared to a state space of 36^8 = 2.8x10^12. Remove a million or ten if you want. I think you're strongly underestimating how large this space is. Almost any change you can imagine pushes people out of the relatively minuscule space of actually-used-passwords and into the vast barren lands of all-possible-passwords and thereby increases time taken to crack. If you think requiring a number and special character are damaging (since people just have them at the end), then how about the requirement that they have numbers and special characters that are NOT at the end? I've seen this before and it forced me to invent a new password instead of reusing one. It's technically reducing the state space, but it increases the space of passwords that are actually used.
Sure, those 27% of people who actually generated a random password and then happened to get no numbers or special characters are annoyed that they have to generate a new password, but everyone else is picking better passwords.
After looking at lists of user passwords <yes>.
User chosen passwords are generally terrible. If you want users to use secure passwords the correct solution is to generate them and send them to users. Granted, users are going to just write them down at that point.
What lists of user passwords were you looking at? I looked through the default one used by John the Ripper (though they recommend using a larger one for any real purpose) and there were hardly any uses of punctuation at all (exceptions "iloveyou!" and "!@#$%^&*") so I can't make the least bit of conclusions. Reading the list of rules it uses to modify passwords, it prepends and appends digits and punctuation but doesn't seem to delete two characters and then replace them with 1!, or anything similar to that. (Though I'll admit to being inexpert at reading the rule formats and not understanding some). That's as close to empirical data as I've got this side of thinking about how I incorporate symbols and numbers.
"correcthorsebatterystaple" is ok, "correct! horse battery Σtaple" is better.
Why are you arguing i shouldn't be allowed to use any key on my keyboard?
There is no reason to disallow any character in a password. let me use them all. limiting to A-Za-z0-9 is goofy and antiquated.
"better" is a dangerous term. Better as for bruteforce guessing resilience? Sure. Better for security? That would be much less certain. Once you discover the users can't enter it on mobile (no Σ on default keyboard) you'd have to provide replacement mechanism for them or allow them to easily reset passwords. Which probably would be the way your security will be compromised soon. And, of course, the users will be losing such fancy passwords at incredible rates, which would generate a lot of noise on "password reset" system which would make it harder for you to detect abuse there. And that not even considering that humans are very bad at avoiding biases and choosing "random" character replacements - they probably would do the same thing in 90% of cases (yes, you are in the 10% of course, but that doesn't change the statistics), like replacing o with 0 and adding ! at the end. Which robots would defeat instantly.
So it's not about forbidding you from using your special characters. It's that using them - on the whole, not for you personally - may not make any improvement, and if you force it, might actually make the matters worse. Encouraging long passwords that one can remember would seems to be actually better. Or doing away with passwords and using something else :)
Anyway.
Yeah, hopefully we'll come up with something other than passwords.
Once you hit 12+ characters it is extremely likely that the password is vulnerable to dictionary/phrase attacks combining whole words - or you're using a password manager.
The xkcd is funny - but it's also a terrible password suggestion.
Don't forget having lot's of passwords written down in an office setting is terrible for different reasons. &$RGJL__"LKHF#3fghu seems great until it's an a sticky or notepad file.
Certainly a passphrase consisting of 4 English words is going to be easier to attack than a password consisting of a similar number of random letters. But as long as you judge a password or passphrase by its information content rather than by its length, xkcd-style passphrases can be just as good as random letters, but are likely to be easier to remember.
Plug: I've written a random passphrase generator, inspired by xkcd's https://xkcd.com/936/ ("correct horse battery staple") that picks random words from a dictionary, subject to requirements you can give on the command line. It optionally tells you how much randomness is contained in the generated passphrase (assuming the attacker knows the parameters you used). Sample output (picking 4 words of 6 to 8 letters):
$ gen-passphrase -v 4 6 8
verity nonfat shunted specking
27405**4
5.6405e17 possibilities, equivalent to:
58.97 random bits
12.55 random lowercase letters
9.90 random mixed-case alphanumerics
9.00 random printable ASCII characters
https://github.com/Keith-S-Thompson/random-passwords
Of course this won't work for sites that restrict the length of a password, or that require special characters.
Ars Technica published an article a few years ago about password crackers, and one of the subjects they interviewed mentioned the XKCD strip in an offhand remark that was flawed and hinted towards what was probably a poor understanding of the subject matter.
Now whenever the XKCD strip is brought up, people who've been shown the article make an appearance and and talk about the scheme being vulnerable while using the phrase "dictionary attack" in a non-specific sort of way.
That's probably what's going on in the GP comment.
http://arstechnica.com/security/2013/05/how-crackers-make-mi...
My alphabet last letter is "ż" not "z".
\w = alphanumeric
Some people don't even have latin letters on their keyboards by default. And "latin letters" aren't well defined anyway. For one thing technicaly "w" isn't a latin letter.
I'm of the opinion that all restrictions on character set in password are unnesessary and harmful. You shouldn't store plain passwords anyway. You shouldn't even pass the plain password from client to server. Why do you care what character set your customer uses?
IMHO the main reason to restrict passwords' character set is to annoy users by forcing them to make another variation of their favorite throwaway password for your criticaly important blog comment section.
> For one thing technicaly "w" isn't a latin letter.
If there was a predictable pattern, that would be even worse!
There's a number of advantages to this scheme:
- It's easy to write down and put in a safe.
- No funny characters required.
- Not case-sensitive.
- If a user makes a spelling error, we can correct it for him without any compromise in security.
- Guaranteed to be free from offensive words, unlike a randomly generated password.
- People are using words anyways, misspelled if necessary, whether we like it or not.
We can also make it even more user-friendly by carefully degrading security, such as trying common word substitutions for the forgetful users, or doing some kind of middle-ground "remember me" that does not let you straight in, but only demands first three words (to keep the family members at bay). Like I said, this required careful calculations as to how much security is sacrificed when we do that.
http://world.std.com/~reinhold/diceware.html
Some intrepid child got internet famous recently for making some money by doing this for you.
This gives you safe strings as well as space-efficient encoding. As a bonus virtually all systems support unicode without any tweaking necessary.
I hope the author of base64k gets around to writing an RFC because it would be a great standard to use.
• Some services require both upper and loweracase characters.
• Some services require punctuation characters whilst others prohibit them.
• Some services prohibit repeated characters.
• Some services impose a minimum password length; some impose a maximum password length; some impose both.
Covering all these situations is nearly impossible.
These restrictions exist for when humans are allowed to generate passwords. This article is discussing systems where humans don't generate passwords and don't in regular operation enter them.
Think of automatic password generation for a secure API endpoint, or automatic generation of private keys secured by a password that typically will be used in a system with it on the keychain but where the backup copies will be stored offsite.
In that case it's acceptable to have a password which is both secure but it has to be easy to type when required but doesn't have to be easy to type (i.e. short) for regular use.
The internet of things means that more and more devices will be creating "users" which are actually just other IoT devices. Those devices/device-users require these sorts of passwords.
Binary password is usually not called a password, it's a key.