The Neural Network Zoo (2016) | Hacker News Reader

The Neural Network Zoo (2016)

asimovinstitute.org

I have always absolutely hated this diagram and think it should go away. I have also never seen somebody who understands the content of the diagram share it as a useful pedagogical tool.

For example, with CNNs, you are building up feature activation volumes based on the entire previous layer. The edges between layer N only connects to two nodes in layer N-1. What are the nodes even supposed to represent? This is not how CNNs works. This explains nothing and is actually just confusing.

This entire diagram should be re written using the block diagrams from the actual papers.

Shameless self-promotion: I was learning about Neural Networks and found the diagrams lacking, so I made a project of visualizing one. Posted a teaser-clip of the project to instagram here:

https://www.instagram.com/p/B4EchudFiTe/?igshid=cuf9ozx0cpc7

Description in the caption, but the gist is that I created a 3D representation of the canonical handwriting recognizer, using Swift and Apple’s Metal framework for rendering.

I agree that these diagrams are very abstract, verging on minimalistic data art. To a large extent, if you know what is going on, you don't need this diagram. If you don't - a diagram does not help.

Vide "Simple diagrams of convoluted neural networks" (https://medium.com/inbrowserai/simple-diagrams-of-convoluted...) in which I discuss various neural network architeture visualizations.

My experience is that this is only shared by people on LinkedIn who have no clue what they are talking about.

To add an anecdote: I’ve only seen it in professor’s labs before and have been looking for it for a long time.

Also in the VAE diagram, the latent space appears to have the same dimensions as input and output.

To me the DCN picture looks ok, in convolution layer nodes receive input only from small area of the previous layer, in general this is true for convolution layers (kernels) and pooling layers. After that it has couple of fully connected layers where the connections go from every node to every node. Or have I misunderstood something in your comment?

Do you know of a resource that has done the block diagram format? I want to understand more about NNs but all I see are node diagrams.

Yes, go to scholar.google.com and search the name of the network. You’ll find the original paper (it will likely have many citations) which will describe the architecture in detail and have a useful diagram for you.

Wow, this article came up at the time and in 2017. And I even see a little comment I wrote back then in the helpful link dang provided.

It looks very different to me now than then. Mostly because for various reasons I actually know what all those networks are. And a fair percentage aren't normally considered neural networks at all (Belief networks, Markov Chains...). Other models are quite old (Kohonen networks, so old I studied them at school in the 90s), other things very broad categories that other classes may or may not fit into (feed forward network, autoencoder).

So the categories are essentially an incoherent mess or a useful cheat sheet for going through the literature, take your pick.

I see this now, where back then I just saw an impressive/incoherent mess and that makes me feel like maybe I'm learning something in my personal research project.

Discussed at the time: https://news.ycombinator.com/item?id=12751585

2017: https://news.ycombinator.com/item?id=15965159

Note that while the article is originally from 2016, it was updated:

> [Update 22 April 2019] Included Capsule Networks, Differentiable Neural Computers and Attention Networks to the Neural Network Zoo; Support Vector Machines are removed; ...

The poster image is also updated.

Thanks for sharing. I remember designing my first neural network, my notepad was full of theses dots everywhere. The dots x lines representation helped me a lot visualizing what is a layer, what is a input, what is the output of that layer.

Reply on news.ycombinator.com