Hacker News Reader

Comment by tzs | Hacker News Reader

Steamrollers, even bigger ones don't learn from their mistakes. Watson does. For example, in one of the auditions they did for the Jeopardy producers, the questions in the category each referred to two holidays which occurred in the same calendar month, and the question was supposed to ask what is that month.

Watson did not get it. it was finding sensible answers, but answers that did not fit the category. It got a couple wrong, then stopped buzzing on that category. After it saw the correct answers from the other contestants for a few, it started buzzing again, with the correct answers.

View on HN

There is a difference between learning from your mistakes and using the same algorithm on an updated data source.

Maybe about this time next week someone will post a thread, "Show HN: I Rewrote Watson in 112 Lines of Ruby as a Weekend Side Project"

Is there? I don't touch things that cause pain. I, in ignorance, touch a hot stove. Now I no longer touch hot stoves, but my algorithm remained the same.

Even if you were right, it'd just be another example of "do submarines swim?". The difference is meaningless.

Having algorithms that learn is nothing new. Steamrollers (neural networks, genetic algorithms, etc.) have been learning from their mistakes for a long time. I think chubot (http://news.ycombinator.com/item?id=2217740) explained best the problem Chomsky seems to have with Watson.

Yes, there is a difference, and it is not meaningless.

Let's go with your example. You don't touch things that cause pain. And you think that hot stoves, fire, etc cause pain. But what if people show you the trick to walking over hot coals? Then you are able to touch an object that you previously would not have. Not because your list of items that cause pain has changed, but because you developed a deeper understanding of WHY they caused pain, and adjust your algorithm to take that into account. You now are thinking about HOW you are touching an object, and not just about the object itself.

To me, this is a clear, meaningful difference. We're not talking about semantics, we're talking about a fundamental adjustment to the questions and processes you go through before making a decision.

I think you're viewing this too simplistically (although I'm about to walk down a mighty simple path myself).

Lets just imagine that we're using NN, and of course ML is large topic, not limited to NN, but this is sufficient for now. Adding new data points changes the weights on the neural network -- or can even add new inputs or hidden layers.

I don't actually know what the hot coals trick is, but lets say that it is walking over them at the right rate -- this info gets added to the inputs, with no burning as the output (of course its probably not a step function).

Basically you've now increased what you know... burning is a function of temperature and duration.

No change to the fundamental algorithm.

To be clear... adding a new data point doesn't just change how you react to that one piece of stimulus, but depending on what context is provided, it can change all the weights and structure of the NN. Effectively, change your world view.

No. Learning is an algorithm that is applied on an algorithm. There is nothing spooky about it. I wish more people realized that before they started claiming that "machines and humans are different in that humans can learn from their mistakes while machines can't" -- of course they can.

If my algorithm is an interpreter and my data is a program to be interpreted, then distinguishing between using a new algorithm and "using the same algorithm on an updated data source" is rather difficult.

Which isn't to say an interpreter is "intelligent" but the criteria of whether you are "using the same algorithm" seems insufficient.

Reply on news.ycombinator.com