Google's Street View computer vision can beat reCAPTCHA with 99% accuracy | Hacker News Reader

Google's Street View computer vision can beat reCAPTCHA with 99% accuracy

googleonlinesecurity.blogspot.com

347 pointsapawloski148 comments

This particular issue (AI performance on captchas) is really quite fascinating. It's an arms race, but the problem is, only one side can win. Google is claiming they have improved their system in some (understandably) unspecified way, but there's only so far this can go. Captchas need to detect whether someone is human, but it has to work for everyone, ideally, even those with disabilities. Any simple task a human can do will eventually be able to be automated. Tasks that aren't currently feasible to be automated, say some natural language processing tasks, have another problem: scalability. To prevent simple databases of problems -> solutions, the problems need to be generated en masse, and for cheap, which means a computer needs to generate the solutions in the first place. And of course, paying people to just do captchas all day already happens.

The street address/book scan approach that Google uses is interesting in that the exact solution is not known, so they presumably have to be somewhat forgiving in accepting answers (as their machine learning might have gotten it wrong). Perhaps this is what their "risk analysis" refers to--whether their response seems "human" enough according to their data, not necessarily whether it's correct.

I don't see a way around this problem for free services that still preserves privacy (so directly using some government-issued ID is off the table). Maybe some Persona-like digital signature system, where a person can go to a physical location with a government ID, and get a signature that says "Trusted authority X affirms that person Y is in fact a person". Obviously this still has problems, as you need to trust X, and it's a big pain in the ass.

There are parallels to the realm of passwords, which are also becoming obsolete (not that there's a good replacement...). Anything that a human can feasibly remember for a bunch of sites is becoming easier and easier for computers to guess.

So basically, computers are taking over the world, and we can't do anything to stop it. God help us all.

> The street address/book scan approach that Google uses is interesting in that the exact solution is not known, so they presumably have to be somewhat forgiving in accepting answers

reCAPTCHA presents a known problem and an unknown problem to the user. If the user answers the known one correctly, it assumes that the answer to the unknown problem is correct too. I believe the text-based reCAPTCHAs will accept an answer within a character or two as correct.

Humans have had an easy time for a while. Good to finally see some competition. :)

On the other hand, why is that we insist on humanity? None of the systems can block a captcha-solving team based in Nigeria. What we need to block is bad behavior by users and not their humanity.

Good point. I imagine that detecting such behavior will also be pretty difficult. It's one thing in a forum, where you'd maybe want to discriminate against idiots (but that would still leave you open to the problem of shills/sock puppets). But for big companies like Google, would they really want to open that can of worms?

Of course, there's a quite relevant xkcd: https://xkcd.com/810/

I think a big problem is Sybil attacks. Let's suppose I make a bot that posts thousands of reports of my computer crashing after using product X. To determine if such a post is bad behavior or not, you have to know if the claim is genuine. That can't be determined by the content of the post itself.

The suggestion to "go to a physical location to get a trusted government signature" has deeper flaws as well.

What's to stop a bot writer from getting a signature and plugging it into his bot?

Blacklisting would soon force him to get hundreds or thousands of signatures, which would slow down the operation.

This said, i don't think "trusted authorities" are the way to go - we've seen the failure of that model with x.509, there is no need to repeat it elsewhere.

Presumably a service would only allow one (or a few) accounts per actual person. The end user will still need some amount of security. I had in mind some challenge-response mechanism, like:

  Person signs government ID number with public key, presents it to authority
  Authority signs public key and asserts it belongs to a real human
  Service presents random number at signup time
  Person signs the random number with the same public key, and presents the key signed by the authority

I'm not entirely sure how that would mitigate people from passing the government signed public key to their bot and letting it take over from there.

That's what I was referring to with services only allowing one (or N) accounts per person. A spammer could use the signature multiple times, but would need new email providers for every single account.

And as an uncle (?) comment points out, there could possibly be blacklists of signatures of spammers shared between providers. That definitely would cause some new problems, of course, but who knows what the future may hold...

Oh I see. I hope it doesn't ever come to that!

Blacklisting the bad signature as well as being able to trace it back to a real person would be pretty significant deterrents.

That seems pretty obvious to me, making me wonder what you're really getting at.

1. Unlike e-mail accounts or such, a botwriter wouldn't be able to get thousands of such signatures to go on as soon as the first one is identified and blocked; if he puts his own key in the bot - bam, you (and everyone else you know) can exclude him from your services permanently, and he wouldn't be able to come back with a fresh account unlinked to that one, ever.

2. The signature is linked to a specific person, so if the private key is stolen, then you (a) revoke it; and (b) punish the thief - because obtaining that signature most likely be treated as a felony, unlike spamming comments or other uses for breaking captcha's.

If multiple signatures from the same key are recognizably the same person, wouldn't this have the unfortunate implication that your identity can be linked across different services? There'd be no way to have multiple email addresses anonymously without being able to trust the provider.

If you're curious about how the algorithm works, here's a recent talk from one of the authors of the paper where he talks about it: http://www.youtube.com/watch?v=iK3HIIsZGJY&t=1m32s

As to your questions, I think the way it's going with these convolutional neural nets, the idea of being able to tell a human from a computer with something like captcha isn't going to work. I'm thinking a captcha that still would work would be a randomly generated natural language instruction that a user has to interpret with some top down reasoning (possibly with a natural text answer). We've still got a ways to go before we can do that really effectively with software.

Ask for x BTC on signup, on the promise that it is returned in y weeks.

I was going to propose that it's only returned on good behavior, but that gives sites an incentives to harshly police their new user population.

Ideally you'd have some sort of service that both parties trust that is holding the BTC.

And then anyone who doesn't have extra resources is prevented from using your service... As if it is a trivial amount, spammers will pay it, and if it is non-trivial, then some people won't be able to afford it. Doesn't seem like a great idea.

MetaFilter is a web forum that charges $5 to create an account. Between that and aggressive but professional moderation, spam basically doesn't happen since a couple of obvious spam comments and you're permabanned.

However, it limits the user population to people with a debit card and $5 to spend. A bitcoin-based solution right now would be even more restrictive, since apart from buying bitcoin in person, you pretty much need a bank account to acquire them.

> It's an arms race, but the problem is, only one side can win.

I don't disagree, but although this specific problem can be automated, the arms race will continue as newer problems (requiring uniquely human intelligence) are presented. We can keep moving the bar until the singularity.

For example, a captcha: "Pick the most unsafe environment", followed by pictures of a house, a park, a volcano, a bed. To solve this, AI would have to have a mastery of language, a mastery of object identification, and a mastery of the metadata of those objects as they apply to human safety.

At the same time, though, a human would have to have familiarity with all of those objects in the context of their environment. A house doesn't look the same to everyone, nor does a park, bed, etc. People's life experiences can also affect their responses.

The problem we would then experience is unintentional prejudice and Western-mindedness in our "are you human" queries, similar to a problem that SATs and other standardized testing methods have been criticized for.

That runs into the scalability problem that I mentioned--someone (say, a Mechanical Turk user) can solve that problem once, and put it in a database. To avoid this, there needs to be a way to generate them cheaply, so you pretty much need a computer that can solve the problem in the first place.

Basically, my point is that there's no such thing as "uniquely human intelligence". There's lots of problems that fall into that category now, but with the continuing rise of computational power, it's getting harder and harder to find new problems of this sort without excluding a lot of real people.

So that begins to look a bit like the RSA problem, doesn't it? Find a problem which can only be solved efficiently given some "secret" knowledge – where, instead of that knowledge (the prime factors) being secret, the knowledge is something it's easy for a computer to use but hard for it to recognize.

I'm probably overgeneralizing, but it's an interesting framing.

It seems more likely that there's no such thing as "computer-identifiable human intelligence". Human intelligence is easily discerned by other humans.

>To avoid this, there needs to be a way to generate them cheaply, so you pretty much need a computer that can solve the problem in the first place.

I don't see how that follows. Many captchas are just text with a confusing grid overlayed - that's incredibly trivial to produce, and producing one tells you next to nothing about how to solve them. Or if that's too easy, throw in some random distortion - still easy to produce, even harder to read.

Or, you could use xkcd's idea: http://xkcd.com/810

While probably not 99% reliability, I think google software could often solve this one too: Reverse image search could provide an image description, which combined with number of search queries for eg 'unsafe $image_description' to give a ranking. Google can already mostly handle synonyms; it does not need a mastery of the area as it can just use correlations and usage patterns.

Also, being solvable for humans and not computers is not the only criteria for a captcha. For example it must be quick and relatively easy for humans, and provide a definitive consistent answer. The brilliance of recaptcha is that it was making the work that humans excelled at actually useful beyond just 'proof of humanity'.

>To solve this, AI would have to have a mastery of language, a mastery of object identification, and a mastery of the metadata of those objects as they apply to human safety.

Actually for a captcha to work, you have to have a problem that is easy for an AI to pose and verify the solution to, but hard for AI to solve. This could potentially rule out some classes of working turing tests.

"Pick the most unsafe environment", followed by pictures of a house, a park, a volcano, a bed.

My sense is that IBM's Watson could tackle this particular question. Also interesting to note that the question introduces external considerations, such as whether or not there's someone dangerous near the house, bed, park, etc. A human respondent will have to ignore the matter of whether there's additional context and second-guess the captcha author to get the problem right -- presumably the most unsafe environment is the volcano.

Of course the simple way around

"Pick the most unsafe environment", followed by pictures of a house, a park, a volcano, a bed.

if you're a spammer is to choose randomly and make four times as many attempts. Which is why catchpas make you type six or more letters giving millions of combinations so you can't do it randomly. To get up to a similar resistance to random attempts with 'pick the unsafe environment' type questions you'd need of the order or eight of them (giving 4^8 or approx 65k combinations). Would you as a user want to fill in 8 of those things?

The key is that you are only presented with one captcha per request. But if you fail too many, you get labelled a spammer.

Like Deep Blue, Watson represented a major advance in the state of weak AI. However at its core, it's still just a text-based statistical search engine. It has no ability to "learn" new abilities. Its algorithms will use previous answers to adjust ranking of answers, but once a category is finished, that data is lost. For new data, its database needs to be re-indexed. For new behavior like recognizing pictures or playing a game, it needs to be programmed.

Watson played a simplified version of Jeopardy with no video or audio questions. This was a key compromise to allow a text based system to compete with humans. The AI problem of recognizing pictures is practically unsolved. It took Google's best researchers, a massive database of cat pictures, and a supercomputer just to train a computer to recognize when a cat was in a photo. So your sense of Watson's present abilities is a little skewed and unrealistic.

Yes its interesting; you can easily imagine that the following trends are detected:

* "home and safe" "safe as houses" etc

* "safe in bed" "safely tucked up in bed" etc

* "volcano" ... no clear data. Certainly dangerous being a volcanologist, but that'd be great AI googlefu

* "pensioner attacked in park" .. "our parks are not safe" etc... Hmm, Parks are the most dangerous.

Because frequency of local newspaper reports is a bad measure for volcanos I guess.

But one requirement of a captcha system is that it be easy for a computer to generate. The system you describe seems to require that humans build up a list of things with obvious-to-human answers, which doesn't scale, especially when they have to be designed under the assumption that "the enemy knows the system" (i.e. a worker on some "library of common knowledge" project could defect).

But if you could have an AI generate those problems, it would be capable of solving them too!

Current captcha systems exploit the one-way nature of problems, mainly "character distortion". This is what permits easy computer-generation but not computer-solution, and does not seem to be present in the class of problems you describe: it's just as easy to go from "noun -> adjective" as "adjective -> noun".

The odd thing with a question like that is how "most dangerous" is determined. Logically the volcano is the most unsafe environment, and people do die on them, but statistically far more dangerous, at least by death count, is the bed.

This is the kind of problem that would still cause issues for some people, especially if you had to scale them to produce identical captcha's relatively rarely (if the same one shows up too many times it can easily be hard-coded, requiring the most basic detection to solve it reliably) in that as you come up with more of these types of questions the expected answer gets rapidly more nuanced.

So it's a captcha that keeps out computers and pedants on HN. Win/win, in my opinion.

Dangerous does not necessarily mean fatal. The posed question specifically states environment, and the bed as an environment is typically a flat, soft, blunt, rectangular prism.

Whereas volcanoes, and more specifically, active volcanoes can be unpredicable and spurt out hot lava and ash.

The question is worded fine and your attempt to poke holes into this is invalid.

Even if a question is worded well, it can always be interpreted incorrectly. The question should not be blamed for a person's inability to process information.

A bot would solve this one with a random selection 25% of the time. You would need multiple puzzles to make this a useful solution.

I can not remember the name of the paper, but researchers have recently discovered that neural networks can fairly easily be disrupted by carefully perturbing the pixels in incredibly slight amounts. Completely invisible to a human, but can ruin the ability of neural networks to classify it.

> It's an arms race, but the problem is, only one side can win.

It's already happening.

It turns out humans come quite cheaply, and you can actually solve captchas using humans at very high QPS for very low cost. And people do!

Site owners will have to come up with alternate means of "bot" enforcement that does not rely on human vs computer detection.

Agreed, cheap labor isn't hard to find, and there are countless people that would answer captchas all day long for a few dollars. How many captchas could you answer in a day? Let's say you work 8 hours, and answer one every 20 seconds. That's 1,440 captchas answered for $3/day. You get 5 captchas for a penny. If you can make more than a penny off 5 captchas and whatever you're trying to post or accomplish, you have a business.

I wonder about the computer's ability to figure out what's spam and to manage the visibility of posts from new accounts dependent on its confidence in that. If captcha proves unwinnable, which I agree with you it very well may be, then all isn't necessarily lost.

All captchas I have seen were used to stop spam. The problem with spam is that 1) it's noise and 2) appears in large amounts.

Captchas approach the problem 1), by assuring only people can post and then hopefully increase the signal quality. It sucks because it requires mental effort, is hard on people with disabilities, blocks potentially useful bots (see reddit bots) and leads to an arms race.

What if we tried to solve the problem 2)? One possible solution is requiring a proof of work. Some computation that requires a few seconds, easily disguised while the user is typing the comment or filling the fields. It's an old idea but I've never seen an implementation in the wild. (yes, it would fail for mobile devices. fall back to captcha or something)

A few seconds of googling gave me this WordPress plugin that requres a proof-of-work when posting a comment: http://wordpress-plugins.feifei.us/hashcash/

Isn't most spamming done by botnets these days, to avoid IP bans? So they have a plentiful supply of CPU cycles?

Also, regular users who visited your site without wouldn't install any software, so they'd be running the proof-of-work as slow javascript, while spammers could use GPU or even ASIC acceleration if proof-of-work became widespread.

It is only a security problem is somebody keeps track.

You can be issued a secure key ('a passport') to make new ids ('HN registration') but nobody needs to know who ('the passport') generated the website specific ids once your key has been used. The passport only gives access to generate further keys.

> The street address/book scan approach that Google uses is interesting in that the exact solution is not known, so they presumably have to be somewhat forgiving in accepting answers (as their machine learning might have gotten it wrong).

In the original captcha algorithm there are two images shown, one where the answer is known and one where the answer is not known. The unknown - the street sign or that extract from an old newspaper - is guessed at by those filling in the captcha.

If 90 people say it looks like '11' and 5 people say it looks like '71' and another 5 say it looks like '77' then you can be 90% certain that it is '11'. The machine has therefore 'learned' what that number is, it is '11'. This can continue being used in the captcha however, now that it is known, people will have to type '11' for it.

That is how it fundamentally works and why the two snippets of information 'to get right'. You can try it for yourself, get the first one of the street sign deliberately wrong and see if it lets you through.

If the recognition algorithms for numbers and faces in StreetView are so wonderful, why are there so many randomly blurred out bushes and bus stop signs? In a typical frame you can find a row of No Parking signs or parking meters and randomly one of them is blurred out, as if they thought it was a face. Machine learning has plenty of learning left to do.

Interestingly I activated a new gmail account today and during the signup process I experienced the obligatory captcha. It was in two parts, the first looked strikingly like a street view picture of a house number, while the second looked like a traditional captcha.

I suspect that google has been using techniques like this to validate their computer vision conclusions. Which makes their 99% assertion even more interesting, because it's likely 99% confirmed by a very large crowd sourced data set, not simply a staff member going through several hundred samples to come up with the success rate.

That's been going on for two years: http://techcrunch.com/2012/03/29/google-now-using-recaptcha-...

Isn't this like free mechanichal turk services? Or forced labour or something. Cynically brilliant outsourcing, tho.

Humans recognize images, machine learning makes use of that in ways no human fully understands, and that makes it "google's computer vision"? AI has really come a long way :P

http://en.wikipedia.org/wiki/The_Turk

I highly recommend watching the TED talk by Luis von Ahn, creater of CAPTCHA and reCAPTCHA. He calls this "human computation". He is also the founder of Duolingo.

https://www.youtube.com/watch?v=-Ht4qiDRZE8

He's also one of the few famous entrepreneurs from Guatemala.

That's fantastic. I was trying to guess from the name. Latin American? German? Vietnamese?

Most likely of German descent. I seem to remember when travelling through Guatemala about 10 years ago that one of the locals that I got to know explained that the wealth in Guatemala, especially around Antigua, is concentrated in a number of immigrant families of European (primarily German) descent. There are about 13 large families and they tend to marry within that group.

It has been like this for a while now. AFAIK you have either

  - the actual captcha and a word from a book (Google Books)
  - the actual captcha and a house number from Street View (Google Maps)

Only the actual captcha has to be typed in correctly.

Do people ever intentionally type in nonsense for the non-captcha part, out of spite, for instance? If you could persuade enough people to provide non-answers for the non-captcha part, would Google stop using it (the non-captcha part)? My guess is it's like spam. As long as it works a small fraction of the time, it will still exist.

I don't think I've typed a whole captcha in over a year. It is always obvious which part it's necessary, and now that most of the time the unnecessary part is to improve Google's bottom line by checking addresses instead of helping digitize books I will continue typing dhgd instead. typing an intentionally wrong word is just as much a waste of time as typing the right word

I did when Recaptcha came out and it was consistently obvious which was the unknown part. It's no longer so obvious, in my experience.

I believe ReCaptcha presents the same unknown value to several users, and rejects outliers.

In my experience, it's still extremely obvious. It's always the word that's less readable (and doesn't look like it came from a book). And if you have numbers, it's even easier.

Recently they added special captchas for untrusted users. If they think you might be a bot, you'll get much harder captchas.

http://googleonlinesecurity.blogspot.com/2013/10/recaptcha-j...

I always refresh until they give me a ``hard'' captcha because the numbers, although easier to read, are harder to type.

Actually often it can be the opposite of what you would think. I don't think Google will distort the "unknown" word as much as the "known" word, making it where the "known" word can be harder to read for a human because it's passed through more filters.

At one time, there was this effort on 4chan to spam recaptcha with "penis" for the second word. When this got to some online news site, they asked the recaptcha guys who confirmed that the traffic was irrelevant.

I suspect the nonsense won't matter. You're drowned out.

I'm sure people do intentionally type nonsense sometimes - I just did for example. Google can get around that by looking for the same answer a few times. Like if the responses are 0000, 4513 and 4513 it's fairly evident which is correct.

All this speculation on whether or not you can type anything for the other word when you can just try it.

http://www.google.com/recaptcha/learnmore

Only worked once for me, and only when I typed a word that was of similar length to the original, but I was only able to get it to work that one time. Leaving it blank or with a period or only four letters would be an extremely obvious thing to check and deny access to.

The 99.8% assertion comes from synthetically generated CAPTCHAs. We don't need crowd-sourced ground-truth -- we know what the true answer is because we generated it.

In effect, Google computer vision got so good that they made their own system obsolete. This is a good thing.

I still think the only reliable way to confirm identity (or humanity) online is an email or SMS verification. Recently, receiving a 2-factor SMS code took less time than the page refresh prompting me to enter it.

That confirms identity, but it doesn't confirm humanity. Robots just need to have a SIM card!

I think it would annoy many human users as well.

It makes it much harder to flood a given site with a large number of actions. For instance, if you require SMS verification for account signup, and you don't allow the same phone number to be used for more than, say, two signups per month, then you make it hard to create a large number of bogus accounts.

Yes, it will annoy some real users, so it's not a no-brainer. But it will put a major brake on many kinds of abuse.

Your underlying hypothesis here is that phone numbers are hard to get. But Twilio only charges $1/month per number, and presumably gets them for a lot less than that. If you use them for multiple nefarious purposes at once, your cost-per-evil could be pretty low.

And that's assuming that you just don't steal them outright. Suppose you boost somebody's Android game, repackage it, and add a little code that intercepts and replies to certain text messages. It'd basically be a SMS botnet.

SMS verification would make certain kinds of casual abuse harder, but I don't think it'd be a big barrier for well-organized assholes. E.g., the "Rachel from Cardholder Services" people that have been using the murkiness of the telephone network to sneak billions of illegal calls past the FTC.

We should just throw a standard turing test at them.

"How was your day?"

What you actually do if the verification is important enough (not often) is ask for a phone number, and have a human being call them. Some e-commerce sites do that, presumably ones exposed to frequent fraud, for example http://www.shatteredcrystal.com/, a game code reseller.

Provide incentive to hack phones, discriminate against people who don't have phones or want to create new accounts.

Wouldn't it be trivial to code up a Twilio-powered bot to defeat the SMS verification?

Requiring SMS or email creates problems for users that wish to remain anonymous. SMS is essentially not an option and the number of free email providers that accept registrations sans-SMS via tor continues to shrink.

99% is probably better than my success rate with reCAPTCHA...

In all seriousness, have there been any good studies of human success rates?

Probably better? I think my success rate is between 33-50%!

Looking at the four examples they give in the article, I'd likely only get one or two on the first go.

- ntrtyLt (I'm mostly confident)

- tmincrw (I'm really confident)

- rrnmtht (I'm not at all confident of my answer, but I'm really confident that r's, n's, and m's should be prohibited next to each other in captchas like this!)

- MCruncy (I'm really confident of my interpretation, but are capital letters supposed to be reported as capitals on this captcha system, or is this one of the systems that we're only supposed to submit lower case characters even when the character that we're viewing is upper case?)

Now that programs are better and better at solving CAPTCHA - that means that correct CAPTCHA input will mean the opposite from what it means now. Since programs are better at solving CAPTCAH than humans, correct input (3/3 for example) will mean it's a robot. Thus, CAPTCHA becomes relevant again.

And how would you deal with the fact that an algorithm could always intentionally provide the wrong answer? It's a losing game.

Presumably humans would see some pattern that isn't there. So you have to get the correct wrong result.

Ooh, that's interesting. I was just talking with somebody yesterday about how his bosses had him spend six months trying to make a new financial system bug-compatible with an old financial system, one to which they had lost the source code. They eventually gave up and just kept using the old system so they could be consistent with previous publications.

Making something bug-compatible with the human vision system might be much harder than pure recognition.

My statement was mostly tongue in cheek. On the other hand, "what does that cloud look like?" would be an interesting captcha.

Oh, and I might be better at the imagination ones than the current crop, especially on my phone.

The endgame solution is to let humans judge other humans. In 5 years, you will be interrogated every time you post to HN. The machines are here to make our lives hell...

That's not the endgame. Once machines pass the Turing test, it will be impossible to filter humans vs. machines.

What if you are interrogated offline? What if the police visits you every 3 months to make sure you haven't turned to a robot?

"The tortoise lays on its back, its belly baking in the hot sun beating its legs trying to turn itself over but it can't, not without your help, but you're not helping."

(http://www.trussel.com/bladerun.htm)

The question is: does it matter whether the participants are human if they are intelligent enough to contribute. I think we'll eventually end up in situation where captchas actually filter out humans :)

Yeah, I've had that weird experience of spambots taking text excerpts and posting it to avoid detection, but it's actually relevant information...

In 5 years, you will be interrogated every time you post to HN

But that's how it already is! :) Eventually spam content will either blend in to be relevant and attractive (we can see hints of it in new youtube ads system, but that's matching to ads only, not generating) or it will be analyzed and scrapped (along with other comments) by other equally intelligent systems. So machines will judge both machines and humans equally I think.

When it becomes impossible to distinguish computer intelligence from human intelligence, do we really want to stop the computers from commenting?

If a computer algorithm can create a more informed relevant comment than a person, maybe i would prefer to see the artificial comment.

From that caption "CAPTCHA images correctly solved by the algorithm", there are at least two of them that I'm not sure _I_ can correctly solve on the first try.

Which is generally my experience with captcha's these days, I only have about a 50% success rate.

CAPTCHA is a failed strategy, time to give it up.

Discussed this recently: https://news.ycombinator.com/item?id=7419667

it's a failed strategy for deterring spammers, but apparently reCAPTCHA has done a good job of training google's computer vision algos. which probably have better things to do than try to beat captchas.

The program solves captcha that I, as a human, cannot solve. Pretty sure that means captcha of that type are definitely dead.

Same here. Over the last few years captchas have been getting more and more difficult for me to decipher. It often takes me two attempts to get them right. Particularly annoying on google groups when you need to add a lot of members and you have to do a captcha for every ten members you add.

Whatever replaces captcha, I really hope it's less frustrating for the actual humans

>In this paper, we show that this system is able to accurately detect and read difficult numbers in Street View with 90% accuracy.

> Turns out that this new algorithm can also be used to read CAPTCHA puzzles—we found that it can decipher the hardest distorted text puzzles from reCAPTCHA with over 99% accuracy.

Am I missing something or could we improve CAPTCHAs by mimicking street numbers?

reCAPTCHA already does this, actually. It's part of the way Google trains their computer vision algorithms to recognize numbers.

http://techcrunch.com/2012/03/29/google-now-using-recaptcha-...

But that's, as you said, to train their computer vision, not to make the CAPTCHA better as a CAPTCHA.

>Am I missing something or could we improve CAPTCHAs by mimicking street numbers?

Do the street numbers also have a lowered solve rate for humans?

Google already uses actual street number images from Street View in reCAPTCHA. http://www.google.com/recaptcha/intro/

To be honest, I can't even solve those reCAPTCHAs on that page (that's one of my biggest gripes about reCAPTCHA). I think we're nearing a point in time where if some(thing) can solve a particularly hard CAPTCHA, we can safely assume that it's not human.

"CAPTCHA images correctly solved by the algorithm" - Ok. Now I have to consider the possibility of being a machine.

Captchas were meant to keep spammers at bay. Unfortunately, that's no longer the case. Thanks to "cloud technology" like DeathByCaptcha - that is, people in countries where labor is cheap solving captchas all day - spammers have no problem getting through reCaptcha-protected sites and forums to do their mischief.

As a result, reCaptcha & co tend to be more of an annoyance to honest visitors than to spammers.

Sounds like a bigger annoyance to spammers. I don't have to go out and enlist a captcha solving team in nigeria.

I have a mediawiki wiki and let me tell you there are tons of old spam bots out in the wild, ones that would fail at the presence of any captcha. Come to think of it, I've experienced most generations of spam bots since I started. With no protection I'd get spam links/registrations every day. With a simple question like 2+2=? I'd get one every 3-4 days or once a week and with reCaptcha I'd get them a little less frequently. I had to switch away from reCaptcha a few years back however when it appeared to no longer prevent spam (people are paid to solve them all day, etc).

Wait, it's not clear to me from the blog post. Did they make a system that obsoletes reCAPTCHA? If so, it's just a matter of time before the spam systems catch up, correct? If so, what's the successor to CAPTCHA? Or is the web just going to be full of spam in the future?

Digging in to the journal article, the technique they use can only scale to captchas with 8 characters or less, so having a longer word is a simple fix.

I've always felt that it would be only a matter of time before computer vision would be able to solve the (re)CAPTCHA problem. Especially since digit classifiers are able to match the performance of humans.

One approach that I enjoyed seeing was the use of reverse captchas. Here you pose a problem that a computer can easily solve, but a human cannot. For instance, if you ask a simple question (1+1=?), but you place the question box off the screen so the user can't see it. A computer would be able to easily answer the question, but a human user would have no way of doing so.

Basically consider why we want to eliminate computers from accessing the sites -- because we want to make account creation expensive, to prevent sybil attacks and giving away scarce resources.

What is expensive? Reputation. That's where credit money's value comes from.

I wrote a more comprehensive piece here, in case anyone's interested: https://news.ycombinator.com/item?id=7601690

My accuracy is way below 99%, good job Google!

Seriously though, I hope this does not mean there will be harder captchas, current ones are already stupidly hard

Here's a captcha idea: make people write a 100-word essay on a specific topic. If it's good, you're accepted and you won't have to do it again. If it's bad, you're either a computer or cheap Nigerian labor. When we get to the point where we can't distinguish a computer from a human, we'll just let them be a part of the community.

Who judges if the essay is good?

Google software could use their 99% successful algorithm to filter potential captchas. Then show the 1% they can't crack to humans.

Now the race becomes who can write the better captcha solver, Google or the spammers? As spammers learn to identify things in the 1%, Google will hopefully improve faster and continue to narrow the "hard to solve" band.

Google are getting too creepy for any sensible persons liking. Addresses which are off the street in apartment complexes are now getting recognized as well.

Whenever I see these kind of captchas I switch to audio captchas. It is rather unethical for Google to use recaptchas in this way.

Reminds me of a hack day at Yahoo where one team made a captcha where you had to match a photo with its tags and another team made an algorithm that would assign tags to a photo. Both based on Flickr humans meant that the captcha was easily solvable by the algorithm.

So, now if you get the captcha right you're a computer, otherwise you are a human?

99% is better than most humans captcha accuracy. Back in my day humans could still beat computers at Chess but nowadays computers can beat humans at Jeopardy and drive. Interesting to see when it fully crosses over.

This is essentially the technology that was discussed here 3 months ago [1], and it links to the exact same article on arxiv, titled: "Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks". [2]

They new addition to the article is that now they have tested the same type of NN on reCAPTCHA, and (perhaps unsurprisingly) it works.

[1] - https://news.ycombinator.com/item?id=7015602 [2] - http://arxiv.org/abs/1312.6082v4.

Folks, I figured it out! Let's use captcha so that visitors can prove they are robots! If you fail these captchas, you must certainly be a human!

It's fascinating how, arguably simple software now, which is the captcha, would inevitably become more and more complex as AI develops.

Ironic how the HTTPS version force-redirects you to HTTP. (Amazon.co.uk started doing this a few days before and it's pissing me off no end.)

Why isn't google releasing the full algorithm?

http://arxiv.org/abs/1312.6082

nice! thx

I'm impressed that their address identification algorithm can solve those CAPTCHAs. I can't make heads or tails of them.

Isn't this expected (and a natural consequence of the fact that it's trained on huge volumes of reCAPTCHA data?)

That is much more accuracy than with which humans can crack recaptcha.

I'd estimate my own accuracy rate to be 90% or less.

I'm sorry, as a human I have had to fill these street view style captchas in all the time for google, so this is hardly a completely artificial intelligence, humans have done it many many times, in fact I'm sure some of the pictures in the articles have come up.

It's a new success for Deep Learning. It seems to be actually 99.8% accuracy according to their paper: http://arxiv.org/abs/1312.6082

That's one order of magnitude higher.

Great... now they are going to get even harder to actually do.

Yet it says I'm a robot a good two of three times.

what if someone has bad eyes of some rare eye problem and can not solve captcha problems at all? in other words fails captcha 90% of times.

But can it beat CRAPCHA? http://crapcha.com/

The crapcha widget actually looks a lot better than reCaptcha. I've always lamented using reCaptcha just because of how ugly that thing is.

I just came here to say fuck reCAPTCHA! I hate it, I can't read it with my human eyes.

And you have added a lot to the conversation so thank you for that.

Google's reCAPTCHA showed street numbers as one of the two catcha-"words" for more than two years.

For me this was quiet annoying to input street numbers of others. It's a privacy issue, it was like helping the NSA spying and one feels bad entering Google's captcha.

What is even more astouning is that Google does not even mention all the croud sourced "volunteers" that trained their OCR database. As Google use an open OCR software (former HP OCR app from '95) it would be a good choice to publish their data back to the community.

I removed Google captcha on my own sites and implemented my own traditional captcha (on the first sight of it about two years ago).

Does google aggregate&correlate data in vision algo?

For example for street numbers they not only have picture of a number, they also have knowledge of all the other numbers on that street and guesses for those other numbers. Easy to guesstimate order of a number by checking neighbouring ones.

Same for book words, they have n-gram database. http://storage.googleapis.com/books/ngrams/books/datasetsv2....

Thats a lot of useful MAP/ML data.

But the example they give for the new captchas all look like random crap, "mhhfereeeem" and the like. Its like they are not interested in structure, just pure geometry of letters/numbers.

>CAPTCHA images correctly solved by the algorithm

well, isnt that great? Because I, HUMAN, can maybe solve _one_ of those (lower right one).

I frickin HATE google Captchas and simply close the page if it wants me to solve one, they are too hard for me.

You're welcome google (we are the rats labs that teached their system how to read)

And the cats, don't forget the cats!! http://www.wired.co.uk/news/archive/2012-06/26/google-brain-...

this is also can be done with tesseract or encog framework... i dont know whats news about this

Damn, that's better than me!

Skynet

This is really Great. AI is getting really smarter and smarter day by day!

Reply on news.ycombinator.com