Hacker News Reader

Comment by marshallp | Hacker News Reader

Google Research and Stanford researchers test out a 1 billion connection, 9 layer, 16,000 core deep learning neural network (see geoff hinton and andrew ng talks on youtube) to recognize 20,000 different objects in images (with low accuracy but huge improvement over previous approaches).

This was done no with no pre-labeled images (except for fine tuning)! A brain that learned from raw images. The same algorithm can be applied to any data type (financial data, text, audio, images/video) without any human involvement (except gatheting of unlabelled data and running the system). Pretty much the artificial intelligence holy grail!

View on HN

"Singularity is near"? "Holy grail"? You may be getting a little carried away here.

The outcome shows a very nice improvement on an unsupervised classification and feature detection task, but it also highlights that unsupervised machine learning still has a long way to go. 16% accuracy from a network with 1bn connections and 100m inputs using (if my math is right) 1.15m hours of CPU time. Which of these would be the easiest way to continue making gains: investing more time/hardware, increasing the complexity of the model, or developing a new and improved algorithm altogether? All of these sound pretty intensive to me.

If the algorithm keeps increasing in accuracy as you scale up computation and add more unlabeled data that is pretty amazing. You might get something that matches human performance on vision/speech recognition etc.

If you extrapolate that way you'd conclude that naive Bayes is the solution to AI. Improvements tend to tail off fairly quickly as you add more data and computation, unfortunately.

I only read the abstract, so I'm sure this is a basic/dumb question... but if you don't label images as faces or not, what makes it a face detector? :) How do you get an elbow detector or a butt detector out of the same algorithm?

It automatically creates a set features that you can then use a final layer of machine learning to get what you want.

In machine learning, normally you have to create a set of features (call feature engineering - basically think algorithms to better represent your data). The amazing thing about deep learning is that the computer does this for you!

You just need a few 10s/100s face/nonface images - same for 20,000 other objects - this is called fine-tuning.

For more, andrew ng, geoff hinton, yann lecun have given talks on this at google and they are up on youtube.

This paper is actually more interesting: it automatically learns some "neuron" which its firing represents a detected face, without any supervise technique. It shows the possibility to extract complex information solely from data.

Is there any concept of "reward" for this thing?

Wouldn't that make training it much quicker and make it much more accurate?

Or are we trying to avoid any human interaction at all with the earning loop?

See, for example this company (one of many) that trains bees to smell certain odours.

(http://www.inscentinel.com/)

Using "reward" or say supervised training is easier and (near certainly) often gives better result, but unsupervised is more interesting as a research result, it tells that we can actually extract very high level information from data itself, using some "obvious" rules (such as linearly mix adjacent pixels and give as sparse-"laplace distribution like" results as possible). It is important because it proves that we may simulate brain functionality without knowing exact structure of brain (as we know brain is complex), but by analysis the data it processes using lots of simple structure instead.

Show it a zillion pictures. then show it a face and see what gets activated. that's your face detector. show it an elbow or butt, and see what gets activated, that's your elbow or butt detector.

Wait until they become conscious and demand human rights. Then we can no longer exploit them and we're back to square one.

Seriously, a Phd thesis not far from now may have the title: "The limits of AI: how far can we exploit the machines before we are limited by machine rights"

I'd say until they day they realize they're being exploited, and complain about it.

And so is a profession of "machine rights lawyer". Not to mention "Corpsicle lawyer" for representing cryogenically suspended persons. Seriously.

Actually, that is not just a thesis at some point in the future, that is an active field of study and has been since the 1980's. Some keywords are "machine ethics" and "AI rights". One of my friends wrote his PhD (in the 'real' sense, i.e. from a philosophy department - just to say that this is not (only) a CS topic) thesis about a question related to this in the early 80's.

Reply on news.ycombinator.com