Hacker News Reader

The world is not a place where events can be sandboxed to one instance like this.

Luis von Ahn did digitize the entire archive of NYT before flipping reCAPTCHA to Google, and I'm sure it's still helping. But because he did that, he had time to start monolingo project (now duolingo) which is helping translate the Web.

So if reCAPTCHA was not being turned into a tool for profit, it would be sitting in dust at some old room in Wean Hall. Or, Luis would be still with it, and Duolingo would never have existed.

Besides, all work that Luis did on CAPTCHA at Berkeley and reCAPTCHA at CMU is published. Nobody is being prevented from making another one just for books. Amazon, maybe?

reCAPTCHA is being utilized for other things because digitizing books in that manner is not useful anymore. Most publishers now have ebook versions of most of their books, which are already digitized.

> because digitizing books in that manner is not useful anymore. Most publishers now have ebook versions of most of their books, which are already digitized.

Perhaps for kids looking for new books that's wonderful, but those of us that use books older than your mom, there are no ebooks. Google Books was literally the only place to access that information outside of a few select world-class university libraries.

I was referring to digitizing books as Luis defines it - actually typing in the scanned pages. Not the scanned pages themselves, as Google Books had.

My comment was regarding reCAPTCHA not being used to convert scanned book pages to text.

For books older than my mom, most of which must be public domain, there are a lot online archives. In fact, some of the more common archives like Project Gutenberg, are available as free ebooks on Kindle and iBooks. For more obscure titles, there are field-specific archives from where Google accessed them.

They only got information from publishers and archives. So information you're looking for is out there somewhere in digital form.

> So information you're looking for is out there somewhere in digital form.

Were that true, you'd think Google would be able to find it.

> > So information you're looking for is out there somewhere in digital form.

> Were that true, you'd think Google would be able to find it.

Or not - because of copyright laws. :-(

> most of which must be public domain

The difficult period is approximately between the 1920s and 1990s, where books are under copyright and may have become orphan works (rightsholders cannot be found).

> For books older than my mom, most of which must be public domain

How old is your mom that a book written when she was alive is almost certain to have been written by someone who has now been dead for more than 70 years?

> Perhaps for kids looking for new books that's wonderful, but those of us that use books older than your mom, there are no ebooks.

Project Gutenberg was a thing before Google Books, and those two things aren't the only source of electronic versions of books "older than your mom", or even the only source of free electronic versions, where such isn't legally prohibited by a combination of copyright law and preference of the copyright holder.

How is reliable Google Maps and Street View not improving the world? I'm using them daily and they're improving my life a lot.

At what cost?

At cost of solving the captcha problem?

They are improving your life in exchange for information about you and the places you visit. Their product is closed, the proprietary data is not accessible outside their apps, or if they wish to license it.

OpenStreetMap is improving the world.

"the proprietary data is not accessible outside their apps"

This is not true. Many APIs are available, specifically to allow building 3rd-party apps: Directions API, Distance Matrix API, Elevation API, Geocoding API, and so on. See https://developers.google.com/maps/pricing-and-plans/

I think you'll agree that there's a big jump from being able to access an API X times a day to having access to a data dump.

There are specific clauses in Google's APIs that allow for things like disaster recovery applications to have unlimited access for free.

So I can download their entire map dataset and host my own copy?

> They are improving your life in exchange for information about you and the places you visit

So they are improving the world and for free. I'm glad we all agree

information about you and the places you visit != free

Not exactly free, but the places you visit can be observed with the naked eye from public property, and a network of cameras feeding into ANPR and facial recognition would be a hell of a lot cheaper (and more reliable) than a mapping product.

Not to mention that realtime traffic data and intelligent routing around congestion is only workable by tracking individual vehicles on the road and recording how long it takes them to get from A to B.

> Not to mention that realtime traffic data and intelligent routing around congestion is only workable by tracking individual vehicles on the road and recording how long it takes them to get from A to B.

Wrong. There is a better solution, used for traffic lights that adapt to traffic:

Use an inductive loop to count every vehicle and its speed at ever lane of every intersection and exit.

Far lower cost yet, and provides far better data. (Also is totally anonymous)

>Far lower cost

Um, what? Lower cost to dig up a street and install infrastructure inside it, than to use the cheap and ubiquitous internet and GPS connected computers that are already in every car?

The sum total cost paid by users of maps in data charges and electricity alone outweigh the cost of implementing a public system. Just because the cost is distributed does not mean it's not being paid.

If you want to compare induction loops against power and data charges then you need to magically make your induction loops capable of giving turn-by-turn navigation without a device in the car.

You need to dig up every street every winter for repairs anyway, and you need the inductive loops for adapted traffic lights anyway.

Effectively, you need the infrastructure, most cities on the world have it in some way – either as inductive loops, or by using RFID toll systems, or similar.

So, you have it already, and you can get the same data out of it without the cost of breaking everyone's privacy.

I live in a very heavy snow area in the North East US (Buffalo region) and we definitely don't "dig up every street every winter for repairs". What is usually done is that when the snow thaws and the temperature warms a crew will come out and fill necessary holes in the pavement, not dig up streets. This process is very fast, it only takes a few minutes per hole and would definitely not include digging up the streets.

Here in Germany we fill the holes every few weeks in winter, and then rebuild the street completely every few years when more than 50% of the street are those filled holes.

So by your logic, unless you're giving something absolutely for free you're not improving the world?

Feeding data into a proprietary system involves the risk that your access to that system may be revoked for any reason by the owner of that system at any point in the future. The threat of revocation, in turn, can be used as leverage to force you to accept damaged version of the system you like, with "damage" ranging from charging money to unwanted surveillance to unwanted advertising to software failure.

Some people think that this risk is unbearable. Others weigh the economic realities and make a case-by-case judgement. The simple truth is that it costs money to host a good tool that's always on, doesn't lose people's data, that keeps up with changes in technology and requirements.

You still haven't demonstrated why they are not 'improving the world'.

They may not be doing it to the extent of your liking, but they are certainly are improving it (if only for the people who choose to use their service).

Demonstrating that was not my purpose.

For the purposes of the argument you failed to back up your position.

I try to use open street map over and over again, b/c it has the better data.

But then their site and UX are so bad, I always cry when I use it for the lost effort of so many people.

Check out http://maps.me/ It's a really nice mobile app interface that uses OSM data.

Thanks will take a look

The main OpenStreetMap.org site is really intended mainly for those contributing to the map. They don't make it clear enough that end users should probably be using one of the many other sites based on their data.

Which is part of the problem. Suppose maps.google.com would only be to report errors and you need to find another site that works.

It's only a problem if you think the OSM project is trying to build a replacement for maps.google.com. Since they're not, it's not a problem. Most users of their free location database are not using it directly on the OSM.org site, but through other apps and services. It is rightly the job of these 3rd parties to deliver excellent user interfaces for the data.

Commercial proprietary products have positive and negative externalities. These can improve the world even if they're not free.

We have am mixed capitalist system of free and publicly funded advancements as well as privately funded advancements.

The internet as it exists today is a demonstrate of that fact, as is the mobile smartphone market. This has obviously improved the lives of billions of people and would not have existed without both private (proprietary) and public contributions.

They're both "improving the world" by some standard, but not quite in the same way. There's nothing wrong with for-profit services, and plenty of "for profit stuff" absolutely makes the world better.

BUT... as technologists / hackers and people with (presumably) a broader vision, I think the onus is on us to prefer and to advocate for services which are "open" or "free" / "libre" (without getting too deeply into the definitions of "open" and/or "free" here).

I would say that it would be wrong to stay that Google Maps doesn't make the world a better place, but I'd also agree with your general sentiment and say let's do all we can to promote OpenStreetMap and the like.

It would be better to contribute to OpenStreetMaps instead of force-feeding their own proprietary platform.

In my country (Romania) it's hit or miss. Not all addresses can be located by Google Maps. Many of them yield just the street, which might be kilometers long. It's a shame that Google Maps is inferior to my iGo tablet running Windows Mobile from 10 years ago. And it didn't need to have continuous wireless data access. Even routes that have been looked up are not saved, so there is no way to recover the route if the app crashes or I need to interrupt it midway and I have no data signal. I am very frustrated with Google Maps as a car GPS. It's an inferior product in my country. How was iGo capable to deliver superior maps so many years ago?

A bit further on the west (in Bosnia & Herzegovina) Google's street view is still not available (not even in our capital). As for the names of the streets, OpenStreetMap is far more detailed (and precise). Even Bing's Maps are more usable.

> It's a shame that Google Maps is inferior to my iGo tablet running Windows Mobile from 10 years ago. And it didn't need to have continuous wireless data access. ... How was iGo capable to deliver superior maps so many years ago?

They purchased data from a mapping company. Ten years ago, the two largest mapping companies in the world were Navteq and Tele Atlas. Google used to buy data from these companies as well, but they stopped paying and decided to build up their own maps data from scratch.

> Even Bing's Maps are more usable.

Navteq was bought by Nokia, got renamed HERE Maps, and has just been sold to a consortium of German automakers.

Microsoft currently has a multi-year contract with HERE Maps as part of their deal with Nokia. Also, Helmut Panke is on the Microsoft Board of Directors. He used to be the Chairman of the Management Board of BMW.

> A bit further on the west (in Bosnia & Herzegovina) Google's street view is still not available

Could it be because of the maze of laws that governs public photography? In the US, it's somewhat simple: what's visible from the street can be photographed. But in other countries, you have murky laws, and Google, with deep pockets, can get sued easily.

In Germany, where Google famously lost, the law is also simple: From eyelevel, you can photograph everything. Above eyelevel, you need permission, or it has to be only imagery that could have been seen from eye level, too.

One reason for this is that people might build a 2m tall fence around their frontyard and walk around naked, but Google with a 3m tall StreetView camera could obviously photograph that.

Google ignored the law, and hoped it would be easier to ask for forgiveness than for permission, but, as Germans are, everyone asking for forgiveness will be punished even harder to prevent such business policies.

From what I've seen, people use Waze in Romania, at least in Cluj. I imagine outside the cities there's probably no provider with decent maps though.

In my region, their data is from 2005, with no MapMaker available, so people always end up in my suburb and ask where the freeway is.

StreetView is only available in some areas, after Google put their cameras at a height of 3.5m, far above eye level, and needing written permission from everyone (cause people built hedges and fences with eyelevel in mind, so people might be in their frontyard, behind a hedge, naked, and now Google photographs that – big no-go). Just another thing where Google thought violating reasonable laws would be possible for them.

In the end, their quality is hit and miss, and the cost is too high. No public transit data, not the ability to download larger areas, etc.

If I worked on making the big mac tastier would you call that improving the world? I mean, plenty of people eat them and improving the flavour improves their lives.

Ultimately the specific terminology is unimportant. Do we agree that there's a distinction between digitizing books in a way that's accessible to everyone, and working to improve Google products that exist primarily to increase Google's revenue, and can you see why people would want to contribute to one and not the other?

First, reCAPTCHA's purpose was to digitize books, not digitize them for public domain. If the work was copyrighted, it was returned to rightful owners.

Second, if Big Mac got 2000% tastier every year, and while at that, 2 billion people got to eat dinner for free everyday, it sure will make the world better.

reCAPTCHA switched away from digitizing books simply because it was no longer a good test for human vs robot -- the robots were better at it than humans.

"[Google's research shows that] today’s Artificial Intelligence technology can solve even the most difficult variant of distorted text at 99.8% accuracy. Thus distorted text, on its own, is no longer a dependable test"

http://googleonlinesecurity.blogspot.com/2014/12/are-you-rob...

But lots of sites depend on reCAPTCHA, it would have been unreasonable to just shut it down and break a huge swath of the internet because it had achieved the "we could digitize just about any book written in latin script" goal.

Data mining books was the lowest hanging fruit. Being a library was a side effect. First and foremost was training their search engine with a massive amount of information.

How do you train a search engine on books? How do you generate relevance metrics (like # of links on the web)?

When it is able to offer search suggestions or corrections to your search, I think some of that was based on machine learning on a massive corpus of text.

I don't think so. I think it simply performs a spellcheck vs existing database of terms and offers a likely correction.

Ie when you put in a word 'Mroscow' it sees that this term has 500 results while a similar term 'Moscow' has 100 million results. Therefore it assumes that you simply misspelled the word and offers the alternative as suggestion.

It does not know what the actual underlying word means.

@Tycho http://norvig.com/spell-correct.html

Perhaps same way people associate facts?

PageRank is one of the 200 metrics Google uses nowadays. It's weight is not as great before.

> identify the images with salads

I tried many times and failed the salad test. It's so frustrating that I gave up.

"Identify the images with spinach" - presents 8 blurry green images of green spinach like leaves.

I failed the "salad test" too, I think because I identified various plates of mixed meat and non-green veg as salads. Probably too much time spent going to hip London restaurants.

IIRC the image recognition doesn't even show up if you just move your mouse randomly across the frame. I have been requested to identify pictures like two or three times in total.

Do you have anything to back this up?

- Google stopped using book scans for reCAPTCHA, because it stopped preventing spam

- How can you claim Google's maps product hasn't improved the world? That's a bold statement. I have a data point: It was used quite a bit during katrina and fukashima.

- The training data that's now being used for ML is helping the world of image and audio recognition immensely. They do publish a lot of this stuff, ya know: https://research.google.com/

> to improve Googles online mapping

then

> nothing to do with improving the world

seems contradictory to me.

Seriously... How this the top comment? It's full of hyperbole and straight up misrepresentation of facts.