Hacker News Reader

Comment by rwallace | Hacker News Reader

Of course you do. You've presented a trivial Python program; an equivalent in neural networks would be a single scalar input multiplied by a single weight to produce a single output. That's just about as easy to understand without having to run it.

View on HN

No, the equivalent in neural networks is a network which is trained to always output "Hello, world!", and I dare say that such a network would be pretty hard to reason about compared to the python equivalent.

Actually, the minimal neural network that always outputs that string would basically have a weight vector equivalent to the ASCII coding of the string, so it would be equally easy to reason about compared to the Python equivalent.

You are comparing apples to oranges. The python line is a good representation of a part of a larger program, while the neural network you are talking about is not a good representation of any part of a larger neural network. You wont get that solution if you train it normally without knowing how to solve the problem yourself. Try setting up a network with 3 layers of nodes and 120 nodes in each layer and train it to output "Hello, world!" using a standard training method, you wont get something easily comprehensible from that.

You're neglecting the emergent effect of neural networks. Fully understanding the output of one or more individual neurons hardly tells you anything about how the network behaves has a whole. From that standpoint, neural networks are nearly impenetrable. With human written software you can at least trace through the source code and get an idea of what it's doing. Imagine being handed a large matrix of numbers, being told it represents a neural net, and then being asked to find out what it does. Good luck!

Having been a programmer for thirty-some years now, I can assure you, impenetrable emergent effects in human-written software are very much a thing, and fully understanding the operation of one or more individual lines of code doesn't necessarily tell you anything much about how the program behaves as a whole. Other things equal, there is some tendency for human-written software to be easier to understand, of course, but it's very far from always true.

Think of it this way: if it were impossible to understand opaque, highly emergent systems designed by nonhuman processes, we might as well shut down every life sciences lab and toss all the biology textbooks into the recycle bin. In reality, of course, we can figure out how biological systems work, if we care to put in enough effort, and artificial neural networks are not only orders of magnitude simpler, but have the enormous advantage that we can run them on digital computers, which at least gives us full access to all the raw code and data.

Yes, but a 100 lines Python program is about as simple to understand as this one. Understanding the equivalent neural network requires advanced tooling and lots of hours of labor.

Now, a 1000 lines Python program is generally comprehensible with a little effort. The equivalent neural network is completely opaque.

But yeah, let's be pedantic.

Reply on news.ycombinator.com