Hacker News Reader

Comment by codeulike | Hacker News Reader

Here's the bit about cats:

If this [huge google] network had been fed thousands of images labelled as ‘contains cats’ or ‘doesn’t contain cats’ and trained to work out the difference for itself by iteratively tweeking its 1.7 billion parameters until it had found a classification rule, that would have been impressive enough, given the scale of the task involved in mapping from pixels to low-level image features and then to something as varied and complex as a cat’s face. What Google actually achieved is much more extraordinary, and slightly chilling. The input images weren’t labelled in any way: the network distilled the concept of ‘cat face’ out of the data without any guidance.

View on HN

I don't think that's as surprising/chilling as people make it out to be. Let's say I run a compression algorithm (e.g., Huffman coding) on a billion strings, and a very common substring is "hello, world". I wouldn't be surprised if the compression algorithm "learned" to compress "hello, world" to a single bit 0. There are a few big jumps to get to cat faces (e.g., the representation has to be approximate), but I don't see why the idea is fundamentally different.

The problem is that this network-contained concept of "cat face" is still a symbolic representation. It's a much more complex algorithmic symbol than the rules found in something like 1960s Eliza, but its understanding of the world is on the same level.

You can't ask the "cat face" neural network anything about cats. It has no idea what they actually are in relation to the world. A two-year-old human can usually tell you more about cats than you'd care to listen.

I think the important part is that it "learned" to distinguish this concept of cat face without that being the direct intent provided through labeling. Given the set of images, it learned that "concept" which we can then retroactively label to make it useful in the typical human sense by associating it with cats. If we had to compose computer "knowledge" by training it in everything by manually specifying what it was training for, that process would surely be insurmountable in a general sense. Yet this seems to provide the possibility of automating this process.

A separate network of connections between "knowledge" bits could maybe be used to associate related knowledge like you talk about in terms of their relation to the world. This could also probably be formed in a similar manner giving the algorithm the ability to distinguish "important" concepts contained in the data. The thing I find most odd and interesting about this is that the network tends to identify different important concepts than humans do.

You can't ask a child's visual cortex to tell you anything about cats, either. But connect that visual recognition ai with something like Watson, and you have what you're looking for, no?

I just don't know. I'd be happy to see that be the case...

But I'm afraid it's going to be the equivalent of this cake recipe: "Take an egg and a packet of sugar. Break the egg over the packet."

You certainly need eggs and sugar to make a cake, but on their own and combined without understanding of the whole, you're not getting very far.

The human brain is mostly a bunch of ugly wetware hacks, not a single coherent 'intelligence' that does all the thinking. If you attach enough single purpose AIs together you might get something much more human like than trying to create a single neural network that does everything.

If word embedding can produce things like "king - man + woman = queen", then concept embedding on images might be able to achieve a similar level of "understanding" (i.e. very low, but probably better than nothing).

I don't see what's "chilling" about it. The neural net looks for correlations among the data, and it noticed that cat faces have a correlation to each other.

How exactly is a neural network called that classifies unlabeled data by itself? Also could they turn it around somehow to produce cat faces?

Google has some software called DeepDream. Basically, you give it images, and it amplifies any features that it thinks it recognizes. If the network is designed to recognize cat faces, then it will take an arbitrary picture and strengthen parts of the images to look more cat-face-like. It's essentially an image-recognition neural network run in reverse.

Reply on news.ycombinator.com