Google Prediction API | Hacker News Reader

Google Prediction API

code.google.com

355 pointspelle81 comments

This is easily the most interesting announcement so far. Machine learning has so many applications, but its use is constrained by the high barriers to entry. Recommendation engines, for example, are huge sales drivers, but few among even the largest ecommerce stores use them. A simple prediction interface that's built on the ML expertise at Google is a win for everyone.

Directed Edge, a promising YC startup, makes recommendation engines surprisingly easy:

http://www.directededge.com/

It's quite a bit higher-level than what Google is offering here, with all the benefits and drawbacks that entails.

Thanks for the friendly plug :-)

You can find the full documentation on our developer site at http://developer.directededge.com/, and we offer a free developer account for non-commercial purposes: http://www.directededge.com/signup-developer.html

If anybody is giving the Google Prediction API a whirl for recommendations, we'd love to hear about your findings!

If I have a graph of customers and what products they've bought, Directed Edge makes it easy to figure out what products to recommend to each customer. Generating the same recommendations with Google's prediction API is not as obvious to me. What data do you train it with? The nth product the customer bought as the output, and the 1..n-1th products as the input?

Maybe I'm slow, but I don't see how google's prediction API is a good replacement for collaborative filtering type recommendation engines.

For very specific problems, yes. The translation classification example is definitely a type of prediction problem that would be good for this service.

However, this is far from a silver bullet to ML problems. It can be quite dangerous, for example, to send off a bunch of data to google and immediately trust their analysis without knowing the underlying application of their algorithm. As a researcher in this area, what I would LOVE is if I could create my own algorithm, send it to Google, upload enormous amounts of data, and get back a result. Because right now it's difficult to scale complex algorithms to datasets in the GB-TB-PB range. Mahout is taking a valid stab at this problem though.

For most people though, I would think a service like this could easily prove to be good enough. What would you suggest that would be a significant performance or productivity improvement over cloud computing?

I don't think Google has some magic "make this algorithm scale to huge datasets" box that they can run things through. The reason why Google can run predictions on huge datasets is because they designed an algorithm that has lower accuracy but greater scalability; this service seems to be just them giving you an API to that algorithm. I think your comment sounds like "I like your algorithm, but I'd rather use my algorithm, so please run my algorithm but make it have the features of your algorithm"...

No, I don't mind using their algorithms (I'm sure they are correct anyways). I just mind that I can't tinker the algorithm to be optimized for my special use case scenario.

You can have accuracy and scalability. It's just that normally you run these types of things in memory on one machine, not on tens to hundreds of thousands of machines. So these algorithms have to be decoupled and that isn't easily done (but I'm sure Google figured it out a long time ago).

The standard supervised-learning stuff doesn't have that high a barrier to entry these days, although offloading it as a service to the cloud is of course even easier. But on the client side, things like Weka are simple enough that plenty of CS undergrads use them in intro data-mining/AI/stats classes: http://www.cs.waikato.ac.nz/ml/weka/

Off-the-shelf recommendation systems are a bit trickier though, yeah.

The API seems pretty simplistic. Does it choose the different supervised learning algorithm underneath without input from the user or does it just use one supervised learning algorithm? Their website does not give any algorithmic details.

The graphic on the front page would be more impressive if the input was "french" and the output was the specific phrase :)

I'd love to see how well it could predict comment ratings from Hacker News.

The following data would be a good start:

1. Text of comment

2. How many points the comment has

3. How many points the article has

4. Time article was posted

5. Time comment was posted

I'd also be interested to see what kind of user bias there is. If you don't provide user names, you could see what kind of rating a comment should have based on its content, and what rating it actually has because certain users are generally loved (pg) or hated (jasonmcalacanis) by the community.

That sounds really interesting! If anyone does the analysis it would be cool to see the results.

That would be interesting. Does anyone have a dataset? Ideally to do this type analysis you would also need to know the upvote (or downvote) history of the users so you can pinpoint any bias, or predict which comments you as a user are most fond of reading.

To be fair, several posters can earn their points on name alone:

- grellas can post a one liner on a law issue

- DarkShikari can post about video codecs

- tptacek can post about security

- patio11 can post about bingo

- edw519 can post a grocery list

There should be one more data point in there, and it's the hard part, because it makes the dataset nonlinear:

6. The same data for parent and children of the comment

Every comment's rating is heavily swayed by its position within the thread. A comment replying to something at the bottom won't get voted on. A comment replying to something at the top, with just as much merit, will almost always get somewhere between 0.5x to 1.0x as many votes as its parent. A child comment somewhere within a low-voted comment's descendants saying "why am I getting downvoted?" has the potential to call off the latch-on-downvoting effect, and thus give the comment a chance to spike upward. A comment in reply to something no one bothered reading will be ignored despite its merits. Etc.

Of course, your "content worth rating" is orthogonal to all of those concerns, and would probably help in changing the patterns mentioned above, which are mostly bad for the comment futures market. :)

Not enough details available on how it works. Would rather build my own at this point. Plus the way this is billed oversimplifies the whole model design process.

Sorry to sound so negative, but I just earned a PhD in Machine Learning. How would you feel if you were replaced by an API? :-(

I know. It's worse than being outsourced.

Next thing we know, the Tea Party is going to start protesting APIs taking our jobs.

EDIT: In seriousness, though, this is a good thing for ML PhDs like you (and eventually me in a couple more years). Building the progress that's already been made in the field into easy-to-use APIs frees us up to work on the next steps.

Now if they just released an Integer Linear Programming API that would give us access to their compute cloud...... :)

Well, expertise isn't only needed for building something, but also for using it.

trust me, you have nothing to worry about. Us noobs with regular bachelor degrees who use this will get into trouble because we can't understand the results and the math is all over our heads. We still need you PhD's...(gives T_S_ a hug)

Thanks man, sniffle.

Actually I'm not too worried about that. More worried that the guys at google are waaaay smarter than me :-)

I was guessing that google (in their never-ending desire to consume more data) would want to use us as guinea-pigs to improve their algorithms. It's not 100% clear to me, but from the terms of service:

By submitting, posting, displaying, or transmitting Data on or through the Service, you give Google permission to process your Data for the sole purpose of enabling Google to provide you with the Service in accordance with its privacy policy. You hereby grant Google all licenses to your Data necessary to process the Data and provide you with the Service in accordance with its privacy policy. As a part of the Service and through provided interfaces, Google may allow you to remotely access, view, and download results of the processing of your Data. (via http://code.google.com/apis/predict/docs/terms.html)

I imagine that they might claim the right to use your data anonymously to improve their algorithms, much like they do for your personal data in their other apps. I mean, what better way to refine their supervised learning algorithms than via an endless supply of training sets? But I hate wading through legalese, anyone have any insights?

Also from the TOS, in the section directly above the bit you quote:

4.2. Google claims no ownership or control over any of your Data. You retain copyright and any other rights you already hold in the Data, and you are responsible for protecting those rights, as appropriate.

Did you miss the phrase, for the sole purpose of enabling Google to provide you with the Service in accordance with its privacy policy? That phrase tells me that the ONLY thing they get permission to do with the data is use it for processing your requests.

And they definitely need that. In order to process requests, Google has to make a bunch of copies of your data, create models, etc. Furthermore Google will need to keep copies so that it can use the model it generated for future requests. If the data that you have uploaded is confidential, proprietary, etc, then this requires copyright permission. (Particularly since in the previous clause they made it clear that you retain full copyright.)

No, I did see that, but to zoom in further, what is meant by "in accordance with its privacy policy"?

For Google's other apps, their privacy policy lets them confidentialize and then use data about you to improve their services. Don't see anything that stops them from doing stuff with your data as part of "providing you with the service", without giving up your ownership of it.

Yes, I'm speculating. But it just struck me as a reason for google to offer this service. And honestly, if I were a user I might be ok with them using my data anonymously to improve said service that I am using.

Using past data to give you better service for future data -- isn't this exactly what you want a prediction algorithm to do?

Need more information. What kind of algorithms are they employing on the backend?

From the very little information that I see available so far, it appears that Google will first stab at discrete predictions. That is, I don't see probabilistic output yet.

Also, from http://code.google.com/apis/predict/docs/developer-guide.htm..., it is clear that they perform accuracy analysis using the training data. That is, there is no "testing" vs "training" dataset distinction at this point; there is just cross-validation of the training set.

> That is, there is no "testing" vs "training" dataset distinction at this point; there is just cross-validation of the training set.

If they just create a test set from the training set, and omit that from the training, what's the difference? The main thing is that you don't want to include the test set in the training step, and I assume they're doing that.

According to what they wrote, there is no separate testing set, so they estimate accuracy based on the training set. They use cross-validation to reduce their overfitting bias. (I could be wrong since I haven't actually used their service, but this is the take-home message of the wording of their explanation. If they intended to convey otherwise, they used the wrong language.)

Darn ... I'm halfway through writing one of these. I guess I just have to make it better..

But wonderful validation of your idea, no?

I knew this was coming.

Were you using their prediction API?

He is the API.

Google should write a FAQ on what happens when their prediction API becomes self-aware

I knew you'd figure out my joke. Have an orange star.

Interesting to see how the spelled-out-version of your joke gets more upvotes. You'd expect hacker news readers to be smarter! :)

Yes, this disappointed me as well.

I had this idea myself, basically machine learning as a service about 3 years or more ago. Somehow, I also knew Google would implement something like this. So while I still consider this a viable startup idea, I knew it would be tough to compete against a behemoth that already has tons of data and experience training countless machine learning algorithms.

But three years ago google didn't have this, you could have cornered the market in those three years.

Not doing something because 'google might do it' is not a very good reason.

well, that's really not the reason i didn't do it. amongst other things (my lack of preparation at the time, primarily) it just wasn't easy for anyone but a few big companies because the biggest requirement is that you have not only scale, but loads of data. this is no longer true. you now have aws, hadoop, etc, so it's easier and cheaper to scale, but this is why i knew google would do it. they had the capacity, and they still do. very smart machine learners, petabytes of data, scale, etc.

The "AI API" is the dream application. Just imagine: you get to implement (and even discover) cutting-edge prediction algorithms, make them scale, and expose them via your protocol-of-choice. No frontends to write, no Joe Luser to support, just beautiful math and hardcore infrastructure engineering.

There are reasons this hasn't been done before. I don't think it's a viable startup idea. Think about the capital you would need to even get something like this off the ground. Doing the AI API 'right' would require a supercomputer, if you take it far enough.

A guy can dream though. Maybe doing high-frequency trading would get you most of the way there.

IMHO, go for the niche: elsewhere in this thread Directed Edge was mentioned, and they are a good example of a business tackling a well-defined problem with exciting tools and techniques.

interesting. i actually envisioned a simple web interface that anyone, including joe luser, could use. the idea was to empower anyone to be more data driven, from the individual business owner in africa, to the small and medium business owners everywhere. i did recently run into directed edge. there's also data applied (their ui is too complex though).

by implementing a web app that anyone could use (mobile or not), i also envisioned a sort of community/market place where people could post their data and do simple stuff and/or have experts try to tackle it for a service fee. and whatever algorithms came out of that would be made available for future data that has similar features. i recently came across a similar site. can't remember the name.

anyway, i know this is not necessarily a viable startup idea. and if it is, it's ultimately all about execution. i'm still dreaming though and was psyched google launched their predict api.

I knew you would make that comment

Are there any (preferably FOSS) libraries that does anything like this?

In Python there is a wonderful library called the Natural Language Toolkit (NLTK) available free and open source at http://www.nltk.org/.

With NLTK you can build classifiers, decision trees, and train/predict with bayesian classifiers similarly to Google's Prediction API examples. It's pretty easy to get started, and it's code that you run locally, so there is no network traffic.

I use it on http://www.protopub.com for classifying rss feed stories based on user feedback, so Protopub can recommend future stories that you might like. NLTK is far easier than rolling your own classifiers, but even that is not too difficult. See the O'Reilly book Programming Collective Intelligence.

There's also Weka, which can use almost exactly the same file format that Google is using, and do the same kind of things (though perhaps with different algorithms). It's pretty pleasant.

http://www.cs.waikato.ac.nz/ml/weka/

I recently used Weka to create a simple rules-based fraud model for Kongregate. It worked very well, and had a lot of options for algorithms. The UI is a little weird, but it's worth checking out.

You scratched the itch I procrastinated to scratch. Others (à la Postrank) tried, you just seem you have tried better! 1 Kudo :) EDIT: Ah, you're the guy behind Raphaël Vector lib, have one more Kudo!

Raphaël was written by Dmitry Baranovskiy. I only made very minor contributions to the library (animation easing, which are pretty ubiquitous in the examples now) and presented the library at the SVG Open 2009 at Google.

I made Protopub to scratch the itch I think a LOT of us have. I am about a month away from a v1.0, and that's when I'll announce it on HN. Until then, I'm tweaking AI algorithms, fixing UI bugs, and making sure the back-end can handle the more than moderate traffic that HN will send. The few users I get from posts like this are enough to do some basic testing.

Super stuff. I've invested in a Dutch start-up that had this vision about 3 years ago, but got stranded along the way and pivoted in to being a website building company because they found they could make more money in the short term that way (mistake...).

Anyway, I have some appreciation for the difficulties you must have encountered, and it doesn't please me but it will please you to know that at least from them you won't be having much competition.

Is it ok to start using your service? (not from an industrial espionage point of view but because it is useful!)

Absolutely you can start using it. I evolve it every single day. I use Protopub exclusively as my feed reader, and have been for about a month.

Right now, Protopub is an experiment, but it also serves as a beacon to other likeminded hackers in NYC, where I live, that I am interested in meeting others who want to create unique and technically savvy projects. It has done a good job of doing exactly that so far.

Weirdest thing, when I read HN through protopub it wants me to log in again ?

edit: hm, protopub.com proxies all requests ?

I wish you the best! You'll perhaps have some competition if it still itches in spite of your job, after Broodwar AI Competition is over though. :)

Can you please try our news recommendation service and let me know how it stacks up against your site in terms of (perceived) accuracy? The site is http://www.euraeka.com I can provide more info if you need it. Email me haidut (at) gmail (dot) com. I just need some feedback from someone who has built something similar/related.

Python's Reverend Thomas library is a naive Bayes classifier library. You can do the language guessing and other basic classifications with it: http://divmod.org/trac/wiki/DivmodReverend

there is nothing out there which would scale to terabytes out-of-the-box

VFDT (Very Fast Decission Trees), maybe?

You can also run those distributively without much problems.

Original paper: http://www.cs.washington.edu/homes/pedrod/papers/kdd00.pdf

You should also look on http://www.cs.washington.edu/dm/vfml/

This is interesting:

"Automatically selects from several available machine learning techniques"

So not only does it learn, it's learning which learning techniques work best for different problems.

As a non-techie, I don't understand the language example they're using. It seems to me many prediction engines are originally built to try to forecast winning lottery numbers or other such gambling events. Google expects me to believe they did this for language?

The point is that they took a large number of documents, which are clearly labelled as to language, gave it as a training set to the machine, and they now have a classifier that lets you input random text and tells you the language it was probably written in.

In principle you can do this with any data set and any set of discrete outcomes.

In general, though, you should expect that the resulting classifier won't give you much insight on why it came up with the answers that it did. Plus it frequently is less accurate than a trained human. But it is much, much cheaper.

Whenever I see the term "prediction" applied to software, I immediately think of the books "The Eudaemonic Pie" and "The Predictors." I guess they've spoiled me.

they now have a classifier that lets you input random text and tells you the language it was probably written in.

Incidentally, Google Translate does this and starts guessing the source language as you start typing. I found it interesting that when you type a single character, w is guessed as Polish, i is Norwegian, s is Czech, e is Portuguese...

Frequency of first-character of words in those languages.

The purpose of the example was to show how the Prediction API can recognise the language used just from a few words. So, the data given to the API is a few words in French, and it's prediction is that the language is French. I believe that they're already using this technology in Google Translator. You just type in a few words and it immediately recognises what language is being used. Then you only need to choose the language to which the text is to be translated.

Are there any input/output samples?

I couldn't find any, but here's an explanation of the input format, with some example snippets:

http://code.google.com/apis/predict/docs/developer-guide.htm...

It's pretty straightforward.

Here's one: http://code.google.com/apis/predict/docs/language_id.txt

Linked to from this page: http://code.google.com/apis/predict/docs/getting-started.htm...

Basically what I see is they implemented an open platform for running classification algorithms that gives discrete categories as output. Automatic selection from multiple machine learning methods - maybe just simple cross-validation.

"Upload your data to Google Storage for Developers, then use the Prediction API to make real-time decisions in your applications."

I can understand the necessity of this, but that'll be some serious lock-in.

Not necessarily. Unless I'm misunderstanding, you're not transforming your historical data in Google Storage. So as long as you kept it backed up outside of Google Storage, then you shouldn't have any issues.

Would you give every data point of yours over to Google?

Data doesn't have to be readable. Often, preprocessed datasets are totally incomprehensible for everyone except whoever prepared it. Multiple fields are combined into composites, rescaled, transposed, etc.

Agreed -- look at their own example data:

wine,5,1,1 no-wine,0,0,10

This data is meaningless to anyone who doesn't know what the columns mean, and you don't have to tell that to Google.

And you can of course replace the labels with any other text you want without affecting the algorithm.

You're providing a lot of entropy to a body that has even more sitting around. If anyone could find some part of a dataset that might be sufficient to deanonymize your data Google could.

Not saying you need to be so paranoid, just that non-readability of data might be comparable to obfuscating your javascript to keep people from prying into your code.

At least to a sufficiently knowledgeable corporation.

This type of myopic anti-Google thinking is getting old. Do your homework, read their service agreements. If you don't agree with them, don't give them your data. If they sound reasonable, assume that Google will respect them. After all, they are not stupid, they understand they cannot tarnish their business reputation by violating a customer's trust. If I'm wrong and they end up being stupid, they can be held accountable by law.

In fact from their TOS (http://code.google.com/apis/predict/docs/terms.html):

Does google use the data you upload for other purposes besides just driving your (private) results via the prediction api?

From their TOS seems like they won't use it for any other purpose. This is discussed on a comment thread above.

I predict this will be a hit.

Reply on news.ycombinator.com