Google bots follow SQL-injected URLs from third party sites | Hacker News Reader

Google bots follow SQL-injected URLs from third party sites

blog.sucuri.net

75 pointscubictwo33 comments

So, this is basically using the Google bot as a proxy. I don't see what Google can do about it, or does Google have to do anything at all.

Just secure your website as if the attacker is not going to use the Google bot as a proxy, because well, nobody can guarantee that the attacker will use the Google bot.

Google bot is nothing more than a computer sending HTTP GET request to your server, so if your site is vulnerable to HTTP GET requests why you write like there is something special about Google bot?

Well it would be nice if google could check the url it was visiting and if there is any sqli in it to not send the request (though this could potentially slow their crawling...)

Why? Do you want anybody but Google to hack your site? Why would Google spend resources on unnecessary detection of SQL injection(which probably will not be perfect anyway and may break legit requests) when anybody can hack your website?

I just can't justify why do you expect Google to spend resources on not running bogus HTTP GET request when anybody can run those? What is different about being hacked by Google bot and being hacked by an unsuspected user who clicks on a bogus link that was put on the same page where Google found that link to your server? Just doesn't makes sense.

> Just doesn't makes sense.

Not only that, but it seems to me that it'd be a more efficient use of resources to spend the time hardening your own site rather than lobbying Google to implement something that only mitigates one potential attack vector. Even then, it just seems stupid because I'm sure there are valid GET query strings that might have select, insert, update, delete, or some permutation thereof in them.

It seems to me that it's just a punt on poor programming habits...

How is Google supposed to check for what is/isn't "sqli"? The proposal reminds me of Yahoo! Mail's old "medireview" problem, where it filtered emails containing the string "eval":

http://en.wikipedia.org/wiki/Medireview#Blocked_emails

Even if you look for somewhat complete SQL strings, if I want to host http://try-sql-in-your-browser.io/?sql=select+foo+from+bar, I'd want Google to index it.

For a hilarious variant trying to protect against "attacks" on humans, use your favorite search engine to search for "buttbuttination".

Awesome, I had seen a few things in the wild like this, but this ended up in some fun reading. Thanks!

Well for one, we could make a standard query value that would say whether or not, the URL is evil.

http://try-sql-in-your-browser.io/?evil=true

It works well for IPv4.

This is sort of sad, now I have to go dig through crawler code to see what we (Blekko) do when we crawl a site. We do avoid known 'traps' (which are often inadvertent) that result in loops but this is a bit more nefarious.

On a related note, if you would like to be the person looking through the crawler code and designing a defense for this, and are willing to work in the bay area. Send me an email :-)

Sounds fun, but I'm not moving to the bay area. :P

How would one differentiate between a legitimate query and one that traps? Seriously, suppose I link to a search result page for MySQL [on some site] for info on such a query, wouldn't it look the same?

Not looking for work atm, so my advice: this is not your problem.

A typical crawler trap is a link to a page which adds some sort of "info" the url, so lets say you have a URL like

    http://not-a-real-host.com/some_page/

And that page has a link for comments which takes you to

   http://not-a-real-host.com/some_page/?show_comments=1

And the designer didn't neuter the show comments link and if you follow it, it takes you to:

   http://not-a-real-host.com/some_page/?show_comments=1&show_comments=1

Ad infiniteum. That is a bug but the same sort loop where the cgi arg is going from ?page=1, ?page=2 is valid.

Basically folks who love working on web crawlers are people who like figuring out puzzles like this. The number of interesting things web sites can do that make crawling them not work is unlimited I believe. We have lots of regular checks (things like how many unique documents have I pulled from a host, or how many pages have the same MD5 hash, etc.) Designing defenses is part of the fun.

He is asking how you plan to differentiate when it is ok for the query to be in the url (perhaps it is checked and validated with a regex on the server so it cant be abused) or when its actually capable of doing something malicious the maintainer is not aware of.

I don't believe this is a problem a crawler should be solving. The site left open SQL injection vulnerabilities, and they're complaining that someone is attacking them. Whether the attack originates from a crawler or not is beside the point.

I agree that crawlers can't solve the problem, the trick is trying to avoid contributing to the problem. Take another well known example, DNS reflection attacks. Clearly DNS can't solve the DDOS "problem" but it can avoid helping out by not recursively resolving queries for untrusted clients.

So any time the crawler can be a bit more aware about what it is heading into, the saves it time not looking at pages which we know apriori we would not be indexing anyway. And that (which pages should be indexed) can only be answered from the set of pages that are crawled.

The article is about an attacker using Google's behaviour as a form of indirection. The attacker can't be traced directly by the vulnerable site if he carried out the attack simply by triggering Google's crawler.

I agree that it mentions indirection, but I don't agree that it is the purpose of the article at all. The article finishes with this line:

> We are contacting Google about it, but it is always something important to keep in the back of your mind. You can’t just whitelist their IP, and allow through without any type of inspection.

I wouldn't be surprised if the authors knew of the sqli vector, and whitelists appropriate clients. The problem is the sqli, not WHO can hit it (or who was ultimately responsible).

Edit:

I just went and looked at the company behind this article. I was too quick to judge apparently. Their business is protecting developers from stupid mistakes like sqli at the firewall.

Therefore, the article really IS about indirection. Sorry.

I think the real problem here is that the author is assuming the Google Bot somehow gets special treatment. Why are you concerned that the bot is sending SQL injection attacks? The same thing could be done by a regular user. Are you defending against regular users? If so, then you're defending against Google bots too. Now if the problem is that your firewall is blocking Google bots, well then you're going to have to let them through if you want Google traffic. It's unfeasible for Google to fix this on their end. Your application should be secure from SQL injection without the need for a firewall anyway, so it shouldn't be a technical problem having to make an exception for Google bots.

The problem is not that the Google Bot gets special treatment, but that attackers get special treatment. It's common to blacklist or slow responses to the IP's of potential attackers. A slow response to a false positive is usually acceptable as long as you have a low rate of false positives, but it can seriously affect your Google ranking if you are slowing their bots...

This could also be Google probing for vulnerable software on your website.

They're doing it for various Typo3 versions (I know, because we got some false positives in the past - Google Webmaster tools warned us about it, we saw the requests in logs, our fault for replying with Status 200 for some invalid URLs where we just showed our main page), they might be doing it for other software where the only way to check for vulnerabilities is to try an actual (harmless) SQL inject.

But why would the attacker want this, how can it benefit them? I don't understand how a third party could make Googlebot hitting a SQLi benefit themselves. I get the cleverness of hiding behind Googlebot, I just don't see how you can "steer" the attack, so to speak, to your own benefit.

There's a long history of being able to search google for sites with vulnerabilities or infections if you know what to look for. With a black hat on, alter a busy forum's code to generate a link spreading the infection via google to every domain that commenters mention; a tiny fraction of those will be infected, those will show up later in a search.

That's a bit weak, and might not pick out decent targets. However, visit the forum as a user, mention some specific sites, and hey presto - google infects them for you, while you disappear in the crowd.

Seems a bit airport thriller though. More likely the leaked sqli and google's use of it were accidental.

Wondering how effective this is for getting a competitor website to block for example the google crawler. Maybe that's the motive.

Reminds me of an anecdote I read on Stack Overflow where a web developer inserted an unprotected database killswitch parameter in his code which ended up being triggered by a Googlebot.

This is a pretty interesting attack vector, though. The old Google bomb being used for more nefarious purposes than spoofing page rankings.

Spoofing IP headers (specifically the source address) is possible.

But the only point in this is to take down a site you won't be able to get into useful information back from this request as the request's response will be heading back to the spoofed address... (though if you are using for ddos it is pointless to get the data back...)

You can't make an HTTP request with a spoofed IP without being in position on the network to do a MiTM. The TCP handshake won't complete, so you can't send the HTTP request.

This would only work on connectionless protocols, such as UDP. UDP offers no message reliability and relies on the source address to respond. This is how DNS amplification attacks work, as the main protocol of DNS is UDP.

Whereas HTTP goes over TCP which is a connection-oriented protocol. TCP offers message integrity by going back and forth between client and server multiple times to verify the message was retrieved successfully. Without a valid source address the 3-Way TCP Handshake used to establish the connection cannot succeed.

thank you, I always wondered but never asked.

Coursera had a great Computer Network course.

I think you can still register and download the videos:

https://class.coursera.org/comnetworks-002/

Oh right, duh.

I think the cause here is google following the injected urls from third party sites.

That's exactly what the article says.

Reply on news.ycombinator.com