Massive Twitter Cross-Site Scripting Vulnerability | Hacker News Reader

Massive Twitter Cross-Site Scripting Vulnerability

davidnaylor.co.uk

83 pointsbyrneseyeview35 comments

Just a public service announcement-- for those of you looking to validate your own site for XSS exploits, you'll find that this list is particularly good at giving you "problem" strings to test against:

http://ha.ckers.org/xss.html

We honestly thought we secured XSS with common libraries on every field for one of our sites, but we paid a contractor to go through and use those strings in various unexpected places and they found a few special and unobvious spots that fell between the cracks.

Don't think you're immune just because you use all the right frameworks and input sanitation tricks. Even the best web developers are going to mishandle things from time to time.

If you think it's irresponsible for Twitter to miss some XSS vulnerabilities, you better be doing this testing on your sites, too, before someone else does and calls you out on it. If you're not testing at least the easy user input exploits, you don't really know how protected you are.

This is good advice, but OTOH, I do think it's going about things the wrong way to a certain extent.

Testing your site with certain strings isn't a complete way to determine if it's safe or not. You should "prove" to yourself it's safe against all attacks. Not just the ones you know about.

Definitely worth trying the "problem" strings as a simple sanity check though.

For example js code:

a). myNode.innerHTML = someUserInput;

b). myNode.appendChild(document.createTextNode(someUserInput));

It's clear from the source that a) is vulnerable, whilst b) is not. I think programmers just need to be trained to have massive alarm bells go off in their head when they see some external string used in the output/markup in an unsafe way.

My point is exactly yours, but with added testing:

  * Do the Right Engineering(TM) to guarantee no XSS
  * Followed by having someone else confirm your certainty.

If you only do the first step, I'm nearly certain your site will eventually be vulnerable simply because we're all human and it's so easy to make mistakes in user input handling.

Otherwise you can wait until someone else proves that you missed something (using similar easy tests). Maybe they'll be nice people who will tell you about it. Maybe they'll be mean and will make you look like a fool publicly. Or maybe they'll be evil and exploit your users without you ever finding out.

Trust, but Verify.

Sure, point taken. I just wanted to emphasize the first ;)

"Screw input validation, love output normalization." to quote larholm from this page

http://www.reddit.com/r/programming/comments/86kgp/xss_cross...

Input sanitation is not the solution to XSS style problems. The author of the article and the people on this page who support this view are wrong.

I would assume Twitter is using conventional ERB templates since their front end is written using Rails.

The ERB templates used by rails don't escape HTML by default--you have to write <%= h user_provided_value %> instead of <%= user_provided_value %> to mitigate cross-site scripting attacks such as this. It's pretty easy to miss, even if you know you should always escape user-provided data.

Django's template engine, for instance, automatically escapes strings (though I think this can be disabled). To not escape HTML, you must either specify it or a method producing said string can mark it as "safe".

Though ERB is great in that it's incredibly versatile--we use it to generate several types of configuration files--I would very much prefer for it to escape HTML by default when asked to render a string in the context of an HTML page generated by a Rails app. Form helpers and methods such as "render" could be modified to mark Strings that contain HTML you wouldn't want escaped as "unescapable", much like Django does.

The way it works now invites problems like this, even from competent programmers.

Ahh, looks like it's being rolled in sometime soon:

https://rails.lighthouseapp.com/projects/8994-ruby-on-rails/...

I think Rails 3 does escaping by default, no?

They're working on it.

Such blatant incompetence astounds me. They can't even validate a field when they know the exact format of the content it should contain?

The thing to recognize about Twitter is that they are blatantly incompetent at everything related to their core business.

It's actually 140 bytes, not characters, except when it isn't: http://groups.google.com/group/twitter-development-talk/brow...

Even after fixing their db/cache/queue infrastructure to not fall over in normal use, they still can't handle a DoS attack at all! They peer exclusively with the highest-latency and least-connected Tier1: NTT/Verio (incidentally, so do fellow idiots OpenDNS).

The business side is in fantasyland, as shown in the documents leaked to TechCrunch. Evan Williams has no idea what he's doing -- Blogger was pure dumb luck, it had already effectively died when it was bought and resurrected by Google.

I don't know how Alex Payne keeps himself sane in that kindergarten.

We're a small company trying to solve a lot of difficult problems all at once. We make mistakes, but it's always interesting.

You also happen to be an miracle worker without parallel :)

See my other comment. The engineer who wrote this code did take the time to validate, but his validation was incorrect.

Call it "blatant incompetence" if you like, but I'm guessing you've written a buggy validation or regex once or twice in your life.

Except that using validation at all in this case is a mistake, especially writing it yourself for a narrow case, much less using regexes to do it.

Escaping, damnit.

Yes, I agree. That's why we've tried to assist Rails Core in reviewing a more comprehensive string tainting model. I've said for some time now that security needs to be institutionalized in frameworks in order for developers to be unable to make stupid mistakes like this one.

If that's bad, remember the old iTunes installer bug?

http://apple.slashdot.org/article.pl?sid=01/11/04/0412209...

They had a shell script which didn't "quote" their $variables. It rm -rf'd entire drives by mistake. It's why I use "$@" now.

So you let the same engineer who writes code green-light it for deployment?

No. We have a code review process. It wasn't caught in review, and was okayed for deployment by another engineer.

The sad thing is people seem to have got used to such incompetence. They expect websites to be broken, insecure, incomplete etc.

The programmer responsible should be made to write out lines or something - "I must always validate every single user input".

What's nuts is how easy this is to exploit. I could explain this to a friend over the phone-- no previous XSS experience necessary.

Compared to the whole mikey fiasco, where he found a hole in the on display script for tweets.

Every non-trivial app has vulnerabilities. One indication of a vuln is not "blatant incompetence"

I've responsibly reported my fair share of Twitter vulns over the years. In general Twitter fixed them quickly. (Except some CSRF/XSRF issue which was non-trivial and was when they had fewer resources.)

Dave Naylor was just irresponsibly publicizing something, which makes him somewhat of a jerk. Being a jerk is not necessarily a Bad Thing and of course we/I don't know the entire story here, but that's what it sounds like.

A fix for this is being deployed as I type.

For those interested in the details: the vulnerability was caused by inadequate invalidation of URLs submitted by users when registering OAuth applications. There was a validation routine on those fields, but the very end of the regex used in that validation allowed any number of arbitrary characters. The test coverage included a variety of invalid URLs, but none that simply put a space after the URL and started a <script> tag.

Just goes to show that you can validate, test, and still not be safe. It took another set of eyes to catch this one.

Unfortunately, the publisher of this blog post did not responsibly disclose the vulnerability to us in advance. We found out about it through other channels.

We would have had the fix live earlier, but as several people noticed in the comments, we had a brief outage.

It's odd that you count completely on validation to catch problems like this, rather than escaping the output as well. Writing regex validations to prevent malicious input seems like a seriously error prone thing to be doing.

Agree. Not trying to give you guys a hard time, but Twitter has had several vulnerabilities caused by unescaped user supplied data being output without sanitization.

Obviously I'm on the outside looking in, but IIRC rendering user pages is not particularly taxing on Twitter, to the point that they aren't cached. Would adding a simple sanitization routine have significant impact?

Normally, we escape output as well. These fields were (foolishly) considered "trusted" and allowed to be rendered without escaping.

Yeah. "We let Twitter (via Kevin Rose) know about this before it went live but we think somebody saw the Test!" is a total numbnuts disclosure policy.

If one finds a vulnerability, follow the rfpolicy[1] or similar. Send an email to security@example.com, not Kevin Rose. (Seriously—Kevin Rose? What the hell does he have to do with Twitter's security response team?)

[1] http://www.wiretrip.net/rfp/policy.html

I agree that it was irresponsible for him to use his knowledge and disclose a vulnerability to the public before giving a heads up to the entity in which the vulnerability impacts. Infact, I am for non-disclosure of vulnerabilities due to the fact that this is how script-kiddies thrive.

I go a step further and assert that due to his marketing background, he used this stint for his own marketing purposes - I guess this can best be summed: to each his own.

Twitter seems to be down now (1:45 PM Pacific)

Yep, it's 503'd for me at the moment. Just thinking of all those poor people who now have news and no place to post it.

Edit Looks to be back.

Yep. 503's. I assume they instituted an air gap.

if this is true, i hope they fixed it by now.

Good thing this did not end up in the wrong hands or we might have ended up with a giant Twitter Worm :)

How do you know we haven't had one already? :/

It still works: http://news.ycombinator.com/item?id=786924

Reply on news.ycombinator.com