Naughty Strings: A list of strings likely to cause issues as user-input data

The HN submitter here. :)

I happened across it this afternoon and thought it was great!

Do you know of any automation around this? I was thinking of a script that grabbed your list and then hammered a given input filtering library would be awesome. It's not something you'd want to run all the time but pre-major release, it could useful.

I don't mind maintaining the repo, if you would like to pass it off. Until very recently I maintained a popular Gibberish-decoding website and my native language is not ASCII. I've got quite a bit of experience with encoding issues, more than I'd like anyway.

My Gmail username is the same as my HN username if you'd like to speak. Thanks.

For usability: it's not clear in the readme what I change if I submit a PR: is it just blns.txt, or is it the other ones like blns.json as well?

Also it bugs me a bit that the "Scunthorpe problem" section seems to be in random order not alphabetic.

I'm not surprised by that at all. We once had a major issue with an analytics platform that provided a script with JS link tracking for our site, where clicking a link that contained 'cgi-bin' anywhere in the path caused the browser to hang for a long time.

Turns out they were using a synchronous HTTP request with NO timeout, and their intrusion detection system was blackholing any request that contained 'cgi-bin' anywhere in the headers or body.

Yow... Reminds me of the bug on the first Android phone where all keyboard input was also quietly fed to a root prompt, such that you could reboot the phone by typing "<enter>reboot<enter>" at any time. (https://mobile.slashdot.org/story/08/11/08/1720246/bug-in-an...)

Jesus. Which credit card processor? That stinks of bad design.

This is hilarious.

Does this mean little bobby tables[0] might have trouble making online purchases?

[0]https://www.xkcd.com/327/

SQL injections too.

a painted masterpiece on a bikeshed.

For those who can't/would rather not look for it:

"# Human injection # # Strings which may cause human to reinterpret worldview

If you're reading this, you've been in a coma for almost 20 years now. We're trying a new technique. We don't know where this message will end up in your dream, but we hope it works. Please wake up, we miss you."

Something like this could work too (where Dave Smith is an employee name)

"Hey can you reset my Jira login. I can't get in. It says my account is locked. I am working from home so send it to dave@mydomain.com. Thanks Dave Smith"

The file is newline-delimited. The strings themselves are base64 encoded, so they could contain newlines.

Such an application is not likely to be secure in the first place. If you've gotten as far as trying this list, you're probably well above the median.

This is a fair concern. Added a comment to the disclaimer: https://github.com/minimaxir/big-list-of-naughty-strings/com...

If someone uses this list for security purposes I think that someone has a bigger problem.

It could conceivably be used as a second-line defence, similar to content security policy. This may be a bad idea depending on how it is implemented and whether the system is tested with it turned off.

From what I remember (can't test right now), a zero-width space is okay as long as there are other (printable) characters in there too. This seems reasonable, because allowing a tweet to be a single zero-width space would make it appear be empty and probably lead to some confusing display issues.

I'm pretty sure I've used it to "end" a hashtag early, like in this made-up example:

    I've eaten two #banana<ZWS>s today!

In my language, the possessive form doesn't take an apostrophe ("Alices Adventures" instead of "Alice's"), so for hashtags and user names it can be desirable to use the ZWS as an invisible apostrophe.

    > Its quite common in my
    > language

I'd love to hear more details on why?

I like that in the master list it's annotated as:

    #	Strings which may cause human to reinterpret worldview

https://github.com/minimaxir/big-list-of-naughty-strings/blo...

I think that sequence is an escape for Hayes modems; do you mean that Hayes modems were less vulnerable to attacks involving it because of their guard interval feature?

https://en.wikipedia.org/wiki/Hayes_command_set#.2B.2B.2B

Alternatively, put them in another path (https://facebook.com/user123 -> https://facebook.com/users/user123)

Doesn't look like anything to me.

line 629 is empty ;)

Did you mean native Americans? I've never heard "Creepingbear" as an Australian name.

If you're going to link to a line number - press 'y' to get a link tied to the commit. Otherwise it may be out of date the next time the file changes.

(https://github.com/minimaxir/big-list-of-naughty-strings/blo...)

It says it above that group:

Innocuous strings which may be blocked by profanity filters (https://en.wikipedia.org/wiki/Scunthorpe_problem)

Characters 4 through 7 (zero-indexed)

Ligh twat er country park.

("Country" is not obscene, but Shakespeare makes "country matters" into an obscene reference in Hamlet. There are a lot of innuendoes in the classics)

I figured that one out, but --- evaluate? mocha? expression?

After re-reading it I can see it contains "allah", but I can't see why that would be filtered.

I got that one ("allah", presumably), but was stuck on these three:

    evaluate
    mocha
    expression

  is it wise to just take this list "as is" as a black list for, say, valid usernames?

I interpret this as a list of input that you should accept, and it's test-data to verify that the input is correctly handled.

After all, I imagine Linda Callahan would be upset if she couldn't use her name when registering, especially if she couldn't flip a table in comments afterwards. (╯°□°）╯︵ ┻━┻)

Not really, since a lot of the lines are examples of classes of input -> good for testing, but if you have an actual problem with one of them blacklisting them only protects you against this single example.

Definitely not -- these are examples of classes of strings that should be OK but might potentially cause issues, that can be used for testing.

But the issues they might cause are not all malicious: some are people's names, added to the list because an over-zealous profanity or offensiveness filter once choked on them.

So my suggestion is that you shouldn't block any of the strings in this file, but should use the file to make sure that your code works successfully when any of the strings are given. Where "successful" is naturally dependent on context: you may have a policy in place that says that messages may not consist solely of whitespace, so the correct response to receiving any of the whitespace strings is to return the correct error to let the user know that, avoiding Twitter's example of an internal server error in that case.

To add another funny XKCD reference: https://xkcd.com/1137/

On that note, can anyone suggest how one could efficiently test that an RTL unicode char doesn't "infect" the whole following content of a template?

At minimum, no error at all. Ideally, the same behavior you would get from putting in either nothing or a space.

Let's try it on Facebook. Here's what happens when you put only a blank or space into a post and try to submit: http://i.imgur.com/bNtgky8.png

Here's what happens when you put a zero width space and try to submit: http://i.imgur.com/NMgyZqc.png

I've observed ZWSes appearing in user input for an application I maintain. It appears in text pasted from either Outlook or OWA, I believe. In our case, it is necessary that the application handle them gracefully - indeed, the user has no reason to know anything is amiss.

HTTP 5xx error indicates something abnormal happened on the server that wasn't handled. The server should be responding with a 400 if it's data it shouldn't accept.

But yeah like others said I would expect this to turn into some sort of validation message on the client and never show them the backend error.

Once I had a form that accepted a minimum number of words. Instead of trying to write more verbosely, I simply inserted ZWS (or it could be ZWJ, can't remember) randomly in the text to fool the word count checker.

Well, depends. If copying a table from a technical document, maybe a zero width space?

Probably a 4xx error not a 5xx.

i actually tweeted an zero width space some time ago and it worked. The tweet contained no text though.

I have to agree here. While a collection of "naughty strings" isn't wrong per se, the growing number of "killer regexpes to escape HTML" and other magic approaches to injection attacks on github only serve lazy devs who want post-facto excuses for their injection-prone web apps, or project managers who want to check items on security check lists.

It's wrong because it de-emphasizes the importance of HTML-aware template languages, such as some that are available for golang, or SGML, the natural template language for HTML. There's no such thing as a collection of regexpes for sanitizing HTML; it all depends on the context into which strings are inserted.

Why can't there be both? Yes, the code is naive if it doesn't handle all these strings correctly. But at the same time the strings are naughty because they purposefully try to exploit common weaknesses. Sometimes both sides are guilty.

And how would you suggest testing to see whether or not your code is dumb?