Google’s self-training AI turns coders into machine-learning masters

My understanding is that Google very much intends Cloud AutoML Vision usable for small datasets. To me "deep learning ... inherently impossible to train small" sounds entirely confused.

Cheers for the assist, but I really did mean that it's impossible to train deep learning with small datasets (also, I'm a "she" :).

yorwba below explains why transfer learning is not a way to train a deep net with small data sets. Rather, it's a way to re-train an already trained deep net. If you don't have a deep net trained on your target domain, then what?

Software devs are used to having access to libraries that make our lives simple. You don't have to invent a new language and write a compiler for it everytime you want to code, because java etc. That is the hope of services like the one discussed above.

However, because trained deep nets are very task-specific and it's virtually impossible for a small developer team with few resources to train one from scratch to reach state-of-the-art performance, developers who choose to use pre-trained deep nets as a library are perpetually dependent on Google, or whatever big company decides to offer its deep nets as a service, to determine the range of uses of their re-trained nets.

The question is, is the power of deep nets constrained to tasks where you have a huge amount of data.

there are a large amount of applications where you can use big models trained on generic tasks where the data is free and available to everyone, and then fine tune to your specific more limited data.

let's say I need a person detector for my cctv camera. I use a pretrained ResNet ImageNet model distributed by Google, that was trained on a super large dataset identifying things like animals, furniture and whatnot on a huge free dataset, and then train it just a bit more on my specific task.

this allows to create very performant networks with very little task specific data.

A typical example is improving ranking in (custom/site specific) search. E.g. a client of a company I used to work at spent a huge amount with them having developers manually tweak search results. To add to it, they had multiple different stakeholders (e.g. sales team, executive team, their customers) who had different ideas about what should rank highly for different queries.

But they had a very simple metric right in front of their nose:

Their "product" was bookings at their customers restaurants. They were paid per cover.

One trivial improvement over the constant manual tweaks would be to simply record the probability of a user choosing to book at a given restaurant if that restaurant is present in the search result for a given term or set of terms.

Once you have that data, you can use Bayes theorem to take a set of tokens for a search and produce a list of probabilities that a given restaurant will be a good match, and rank accordingly. And you don't even need to understand Bayes theorem (though as someone who usually don't like maths papers, Bayes paper is remarkably understandable with very little mathematical background) - you just need to be aware of it and be able to use a simple library.

[you'll want to do some tweaks to account for the difference that position in the result makes]

There'll be an endless stream of improvements and more advanced methods you can use, but the beauty of applying Bayes theorem to data like this is that it is very simple, and you can explain to people why it ends up ranking the way it does very easily (the action you record, such as the decision to make a booking, serves as an "upvote" effectively).

There are lots of more advanced approaches you can take if you have the time and the skill and sufficient data, but the above works reasonably well with very little effort and is very often a substantial improvement over whatever manual attempts people make at guessing how the ranking ought to be. And it can be applied not just to search results, but to pretty much anything where you can derive an input set of terms (be it e.g. categories, or a paragraph from the text on the page) that has some likelihood of having some relation to how people will like another item, and where you can record an action against a related item as a "vote" for that connection.

One simple method that is often good enough is k-nearest neighbors. Basically computing the k most similar training values to a test value and then averaging their outputs to get a prediction.

It's cool because it can train on very small data sets and recognize multiple images in one frame. For example, draw a chair, then draw a lamp next to it. It will recognize both the chair and the lamp not having been fed millions of images of chairs and lamps together.