Hacker News Reader

Comment by domlebo70 | Hacker News Reader

Let me warrant this by saying I don't know the first thing about how neural networks actually work, but can someone help me out and explain why the following isn't working?

I added an extra input to the training data for the XOR example:

  net.train([{input: [0, 0], output: [0]},
             {input: [0, 1], output: [1]},
             {input: [1, 0], output: [1]},
             {input: [1, 1], output: [0]},
             {input: [3, 4], output: [7]}])

But the result of running:

  var output = net.run([3, 4]);  // [0.987]

  [0.9999823694382457]

which strikes me as odd. Shouldn't it be closer to 7?

View on HN

From the documentation:

    Each training pattern should have an input and an output, both of which can be either an array of numbers from 0 to 1 or a hash of numbers from 0 to 1.

Not sure about the implementation, like is .train() auto scaling the network size? I'm guessing there isn't enough neurons to do 3-bit xor logic. Try adding more hidden layers!

Also keep in mind that there needs to be neurons to convert a value into binary, and back! Which is also going to take a lot of neurons, you could try doing dec -> binary conversion before sending it to the neural network:

    net.train([{input: [0, 0, 0, 0, 0, 0], output: [0, 0, 0]},
               {input: [0, 0, 0, 0, 0, 1], output: [0, 0, 1]},
               {input: [0, 0, 1, 0, 0, 0], output: [0, 0, 1]},
               {input: [0, 0, 1, 0, 0, 1], output: [0, 0, 0]},
               {input: [0, 1, 1, 1, 0, 0], output: [1, 1, 1]}])

    var output = net.run([0, 1, 1, 1, 0, 0]);  // [ 0.919125109533231, 0.9195887654595207, 0.9734227586511985 ]

outputs of vanilla neural nets are in the range of (0,1). Usually the last step of the neural net is to take the final output and transform it using the logistic sigmoid function, which takes values from all real numbers and compresses it to that range.

https://en.wikipedia.org/wiki/Logistic_function

If you want to create a neural net that can, say, identify handwritten digits and output a result 0..9 it is usually better to create a neural net that has 10 outputs, each corresponding to one digit, rather than a neural net that has one output, say, parsed between (0,0.1) = 0, [0.1,0.2) = 1, etc.

Reply on news.ycombinator.com