Hacker News Reader

Comment by dr_zoidberg | Hacker News Reader

Hinton explains the concept of capsules in this video: https://www.youtube.com/watch?v=rTawFwUvnLE

Which is a lot better than reading someone tell you about this new idea that is called "capsules" but doesn't go into detail. The only thing is that, when this presentation was given, it seems they hadn't worked much more than MNIST (so the new thing now would be the toys-recognition net).

Better source, with date: http://techtv.mit.edu/collections/bcs/videos/30698-what-s-wr... (December 2014, for the lazy).

View on HN

Thank you for the link to the video - this was excellent. His key critique of convnets is that (paraphrasing) all of these pooling layer (max, avg, whatever) are really a very poor way of attempting to solve what is actually a routing problem.

at a first glance this looks like it's going back to old 80s research on neural representations, back then this kind of stuff was known as Parallel Distributed Processing

https://mitpress.mit.edu/books/parallel-distributed-processi...

Indeed. A blast from the past.

Oddly the capsule approach is how I naively thought image recognition worked until I learned more about it.

The only thing is that, when this presentation was given, it seems they hadn't worked much more than MNIST (so the new thing now would be the toys-recognition net).

The paper however details their experiments on Cifar-10 and other datasets in the discussion section.

They don't produce good enough results but the paper proposes certain hypotheses for the poorer performance and that it could be overcome in the future.

Arxiv-link: https://arxiv.org/abs/1710.09829

Apparently, one of the main changes since the talk, is that then he used a "probabilistic unit" next to his "property vector". Now the length of the property vector indicates the probability.

In their second publication they actually use the probabilistic unit, but computing those requires running an EM algorithm for each layer of capsules: https://openreview.net/forum?id=HJWLfGWRb

Reply on news.ycombinator.com