Hacker News Reader

Comment by amelius | Hacker News Reader

I was thinking, can we invent a more formal, common graphical representation of how neural networks work, just like how we have a graphical representation of electronic networks?

View on HN

We do. If you’re interested in the architecture particulars (I.e. which layer types, layer shapes, layer parameters, activation functions, etc) you’ll commonly see these in deep learning papers as a directed acyclic graph with layers represented by a collection of vertices or rectangles. Other architecture specifics are sometimes noted with text alongside layers. We also have more general representations of deep models which look like, and sometimes are exactly, Probabilistic Graphical Models. PGMs have their own formal language which makes it quite easy to write complex models describing a joint distribution very simply.

However, this is different than interpreting how a deep model works. “Interpreting” is an overloaded and poorly-defined term of active research. The sense I get from my friends in the interpretability/explainability research world is that despite the buzz, there is no common definition that lends to a clear set of requirements for an acceptable interpretation of a neural network.

One of the major selling points of neural networks is that you can model/learn relationships that are very hard to explicitly write down. Also keep in mind that neural networks often work with floating point values to scale input values. Sure, you can write down these floating point values in a diagram, but the numbers themselves will not tell you why they have the magnitude that they have.

With that being said, there is research happening that tries to find better tools to understand the reasoning of neural networks. And sometimes we might actually see and understand a general decision pattern of a NN that we could adopt for a simpler model. But in many cases it will not be possible to come up with a simpler model because some phenomena have an intrinsic complexity.

On a related note, there's a "Interpretable Machine Learning" approach that tries to provide more insight into the "whys" of ML. (I'm not en expert and I've learnt about this quite recently)

Reply on news.ycombinator.com