To provide some feedback:

1. display weights and gradients (in addition to activations)

2. display how activations/weights/gradients change during training. For example, I should be able to point to checkpoint directory where my model is saved every epoch (or every few iterations), and then hit 'play' button to see how a particular feature map or weights filter is changing as I move between epochs/iterations.

3. display which inputs contribute the most to the activations, at every layer.

These additional features would be very useful for debugging, for model compression, for identifying information flows (e.g. in a DenseNet), for saliency analysis, and probably for other things as well.