Google’s AI Wizard Unveils a New Twist on Neural Networks | Hacker News Reader

Google’s AI Wizard Unveils a New Twist on Neural Networks

wired.com

31 pointstosh8 comments

Lots of discussion last week:

https://news.ycombinator.com/item?id=15609402

Kant's notion of synthetic a priori knowledge, that is, the innate abilities for geometry in the human mind, the preprogrammed understanding of space and time, seem somewhat related.

The mystery remains for humanity itself: How can we have synthetic truths, like E=mc^2, without direct experience? We may have experimental evidence that implies the formula is correct, but nothing in nature will directly confirm the simple and elegant expression of such a mathematical formula.

In an unrelated note, I think HN should have or at least suggest that submitted links remove referral tags like the link to the article.

It is a perfectly valid reason to flag the article.

So, what is a "capsule" exactly?

(please provide a textual explanation, not video)

You remember the semantic web and what it was supposed to accomplish (...describing and linking every entity possible to create a new concept of understanding and knowledge utilization in the digital space)? It was all the rage ~8 years ago and mostly flopped.

Capsules are kind of like that effort, for the current AI push. They're meant to make it possible to understand smaller entities and how they pertain to a larger more complex entity. The smaller entities can inform as to the identity of the larger entity (particularly in combination, and increase or decrease accuracy depending on the combination), without having to have a perfect or clear picture of that larger entity.

In the semantic web, a complex entity might be cross-linked to a dozen or several dozen smaller entities that help describe the larger entity. With enough of the smaller entities (attributes of the larger entity) you could map and narrow down what the larger entity possibilities are.

As I understand it, Hinton has also referred to capsules as "thought vectors". (I think) a capsule is an encoding of information in a persistent mobile form that allows it to be reused in other contexts / nets / knowledgebases -- like a word2vec word definition vector which contains the scores of the most informative n (principal) component words for the given word.

Another obvious attraction of capsules would be the potential to recognize, tune, and maybe even manually introduce labeled concepts in nets during training -- perhaps enabling one shot learning?

Another obvious attraction of capsules would be the potential to recognize, tune, and maybe even manually introduce labeled concepts in nets during training -- perhaps enabling one shot learning?

This is not obvious to me at all. Can you please explain?

Reply on news.ycombinator.com