There is a bit over 1B passwords in there (based on the size of the file and the length of the line). You would need a binary file around 3GB in size that would have to either load into memory or do about 17 accesses to read specific bytes ( no searching) to figure out if the password is in the filter.
* https://scotthelme.co.uk/when-pwned-passwords-bloom/
* https://scotthelme.co.uk/sketchy-pwned-passwords/ – here he uses a count-min sketch to also store the frequency
If I had the ability to download a massive file I’d try it out on a hextree I toy around with occasionally.
If you’re making an index file may as well just throw it into a tree structure where a lookup is anywhere from 1 to 20 pointer dereferences (assuming the checksum is 20 hex digits) as it optimizes storage so tree depth is variable. Plus it can retain the counts as well.
Now I really want to try this out, the last article I read along these lines I used it as a comparison and it was equally as efficient as their conclusion.
If I served a site where people could check if their passwords leaked I would not worry if one in a million viewers got a false positive.
plonk