Do you really feel like visualizing Neural Networks helps to understand them better? I have yet to find one that has helped me understand it any better than a textual explanation or pseudo-code of the algorithm.
If the learning rate is too high, you can visibly see see weights flicker between different colours. In simple nets you can use it to identify the "meaning" of feature detectors by observing positive and negative links (green and red). You can debug learning algorithms by immediately seeing if something unusual is happening to the weights or activations.
As always caveats apply, but it is a useful technique (when used alongside a variety of other tools).
When I was in school, I learned that a Perceptron is an inequality represented by an hyperplane in the space of R^n (for n inputs of the cell). The numeric output of the cell can be interpreted as the size of a vector perpendicular to such hyperplane.
At the time, I tried to make the next step for a Back Propagation network. Assuming the output in each cell in a given layer is a dimension in the inputs of the next layer. Then I could project each hyperplane onto the other plane to find the non linear geometric objects that resulted. Of course in order to do that, the network structure was constrained to be 2-2-2 (so every hyperplane would map to a line in 2-D).
I lacked the math background to make it work properly at the time, but if someone is interested in picking up the ball, i think it'd be an interesting idea.
https://www.youtube.com/watch?v=ZmNOAtZIgIk
The example starts at about 18:20. You really need to have watched the previous material to understand it, but the basic idea is that he's plotting the hierarchy of features learned by successive layers of a deep neural net (a sparse autoencoder, IIRC, it's been a while since I watched it).
I always run into trouble when visuals encode too much information in colors... I'm red-green color deficient (commonly called color blind), so discriminating colors that have reds and greens as components is challenging.