Hacker News Reader

Comment by bdamm | Hacker News Reader

This part is very good advice:

  When processing requests to establish and change memorized 
  secrets, verifiers SHALL compare the prospective secrets 
  against a list of known commonly-used, expected, and/or 
  compromised values.

So, no rotation, no character set rules, but the password better not be already known. They specifically refer to comparing against a corpus of known broken passwords. Very good.

View on HN

I wrote a tool a while back that's excellent at this. It produces a 1MB file that you can do a straightforward search against and use to blacklist 161,000 different common passwords.

https://github.com/robsheldon/bad-passwords-index

I want people to use it. I gave it the most permissible possible license. Please people, use it. I'll even update it Real Soon Now.

> There are no fancypants libraries, frameworks, or nonsense required. Use your favorite language with the index, I don't care what it is. ...unless you like XML. If you like XML, I'm judging you.

My hat's off to you, good sir.

"bad_passwords.txt" seems to be 1 line long? Am I missing something..

edit: Oh, "Convert your user's password to lowercase and do a simple string search against the index. No regular expressions are necessary." That makes sense, apologies.

I like your license choice. Is there a reason you chose Unlicense over CC0? I've never heard of it before. The text is a bit shorter than CC0 which is nice. Creative Commons recommends against using CC licenses for software so as not to step on the toes of the GPL which doesn't seem applicable if you really want public domain. [1]

[1] https://creativecommons.org/2011/04/15/using-cc0-for-public-...

Oof. I don't remember exactly, but I do remember spending a lot of time (too much time) deciding between various license options. I think in the end I chose Unlicense primarily because it's easier to find Unlicensed projects on Github.

Why not make a Django-specific version, like

https://github.com/timboss/Django-Common-Password-Validation

Because then it would be Django-specific.

There are a few approaches that would work better for specific environments. I could make a gigantic Bloom filter for example, with lookup functions in a few languages.

But my goal was to make something that any programmer could use in any environment. Substring searching is a basic programming skill and just about everything supports it. (COBOL74 needs external functions to do it.)

No, I meant in addition; if you want it to be used widely, a few tweaks (if that?) and you could make a Django version (Django already supports the feature, it just uses a short (1000) wordlist by default).

So https://github.com/timboss/Django-Common-Password-Validation is just a a longer (10,000) wordlist, for example. If nothing else, it makes what you worked on more discoverable.

Out of curiosity, what packing algorithm did you use?

Calling it an algorithm may be giving it too much credit.

The packing program reads in a file, sorts the passwords into clusters based on their length, maintaining a popularity count as it goes. Then it prunes any passwords that occur less than some number of times in the input file ("4" by default, so it doesn't spend a lot of time churning on unpopular passwords). Next, starting with the longest passwords, it works its way down the list, merging the popularity count of shorter passwords into the most popular longer passwords which contain the entire shorter password. e.g., "password" gets merged into "password1234".

Finally it starts writing the output. Starting with the most popular password, it searches up to N passwords ahead in the list (40 by default), looking for any passwords that begin with the same characters that the current password ends with. So it finds "123456", and writes "password123456" to the output file.

I think the only thing I can do to optimize it further is to re-cluster the passwords by their starting character during the output stage, to improve its chances of always finding two passwords that can be glued together.

It's a crude brute-force approach but it works okay.

I just looked at the code, it's not that bad, I really will release it with the next update.

What I'd do is sort the popular passwords by length, start with the longest. Then as you're adding new passwords to the end, check if it's already in there. Then you'll go from phrases to words with a lot of them being duplicates.

At least if you wanted to keep it as a long string. I'd still probably approach this as a bloom filter as others suggested since it'd let you have a larger number of entries for similar memory footprint.

Good, but might I suggest a larger set via bloom filter.

A Bloom filter is absolutely the right way to do this sort of thing, except that I had as a firm requirement, "something that any programmer of any skill level can use in any environment."

The file is small enough, and the approach straightforward enough, that for instance somebody could write a trivial Wordpress or Joomla or Drupal plugin for it, and they wouldn't have to think about Bloom filters or have ready-made lookup functions.

I placed greater value on adoptability than cleverness.

Thank you for this. I was looking for something similar the other day.

Yes, it is simple, yet elegant solution to the common password problems. I would also add check against common patterns (numbers or letters in increasing/decreasing order or keys subsequent in keyboard etc.) to this, so you can't use "password1239" instead of "password123" and call it a day.

I'd say to just treat any of the common passwords as a single character when validating. If "password" was treated as a single character, then password123 would only count for 4 characters. It might be tough to explain to users, but it would solve the problem.

Don't worry, we will find a way around that stupidly draconian rule by making it not look like a word.

For reference, common English dictionary words have about 24-bit entropy, a character has some 6 bits.

That sounds much too high. What common dictionary has 16 million words?

And which user actively uses more than a couple of thousand words in something like a password? I'd expect actual entropy to be way less.

How long before this advice triggers someone to create a service to test passwords against a known corpus with a nice API, and, worse, people using it?

Need to find the URL again, but this already exists. Yes, WTF indeed.

Best idea, send a password over the Internet to verify it. (sarcasm)

Even with good TLS it is a stupid risk to take when you could verify locally. (Just the DoS potential not to mention compromise risk.)

You do realize every time you login to a website you are "send[ing] a password over the internet to verify it" ;)

The same got my attention too. A really great idea, though I'm not aware of it being used currently at all. It might be, but I've never come across/heard anyone mentioning it. Hopefully this takes root!

Actually, couldn't privacy-caring service providers also check current passwords against newly-leaked lists in case something pretty big gets disclosed and warn their affected users? Would probably take some time for services with big amount of users, but that would be some awesome help to users who use same passwords in most places.

Not sure if they still use it, but Tumblr blocked setting your password to a common password in the past.

(Source: I wrote the feature)

Cracklib has been around for ... years.

https://github.com/cracklib/cracklib

I've been implementing this on secure sites for about ten years.

Yeah, I think this comparison against known/expected/compromised values is the key offset for not imposing character set rules. They made the smart choice.

An easy to overcome hurdle:

Pass1word

Pass2word

Pass3word

(my scheme at my last software gig when IT required periodic password changes)

Reply on news.ycombinator.com