Hacker News Reader

Comment by krackers | Hacker News Reader

Why only street signs though? The new captchas I've seen ask you all sorts of random image recognition things, like identifying the squares with grass, with tea, with eggs, etc.

View on HN

Pardon me if this is stating the obvious, but the captcha images being decoded serve a second purpose. In short, Goggle is crowdsourcing the decoding of those images in a mechanical Turk sorta way.

Why street signs and house numbers? Because when your taken as much Street level photos as they have some are gonna need human review.

At first, ReCaptcha was used to transcribe books, and then in 2012 Google began to use it for street data. The GP is referring to its newest use, which is based on an image classification algorithm. Image search used to rely on nearby words on websites and in the URL, since computer vision was just a research technology.

Ironically, this new captcha has already been attacked using Google's own reverse image search: https://www.blackhat.com/docs/asia-16/materials/asia-16-Siva...

A possibility to break these might consist in using Google's image reverse search and check whether the word which you are supposed to identify images for appears on the result pages. However, the Google search is protected by reCaptcha - so you could build your own image reverse search or you use one implementing captchas which are easier to break.

Otherwise, if you own a highly frequented website, you could ask your users to solve the captchas for you by proxying the challenges to them.

For the "which image is..." type captchas, there is always an option to "refresh" the captchas and get a different challenge. After 1 or 2 refreshes, you almost always get a street sign challenege. I was playing around with breaking Google's captchas a few weeks back and tensorflow's basic image recognition demo is actually already good enough that it too can almost always correctly identify which of the 9 or 12 images contain street signs. So I think that's a good start- exploiting the fact that Google favors showing street signs, and that in many cases you can already programatically identify street signs with image recognition. Obviously it's not perfect or Google wouldn't need to farm users to aid in improving image recognition , but it's good enough where you can break the captcha roughly 50% of the time.

Reply on news.ycombinator.com