Hacker News Reader

If you try to design an algorithm, what you get is a model of how you think your system behaves. If you train a neural network, what you get is a model of how your system actually behaves. In other words, a neural network does not embed any assumptions or preconceived notions of how the system ought to work. Intuitively, not incorporating knowledge of how the worlds works ought to be a disadvantage, but in practice it turns out to be not just an advantage, but an embarrassingly large one.

Neural Networks aren't the only common approach in modern AI research, but the other techniques (e.g. Support Vector Machines) have similar properties: they're dumb methods of finding correlations in data. The field of AI is slowly coming to grips with the fact that more training data trumps better algorithms.

That said, I sympathize with your frustrations. You can solve a lot of problems by throwing more cycles at it, but you don't learn anything, either about the algorithms or about your target domain. For example, a human plays the game of Go differently than a machine. A human has an intuition of what moves are good and what are bad, and only considers a handful; a machine relies less on evaluating board position and more on brute-forcing through possible board configurations. As machines get more powerful, eventually the machines catch up to humans in ability, with that same approach. While their methods will become impressive in their efficacy, we learn nothing about how humans are so much better at evaluating board position and selecting moves to speculate on.

Sensing is easier to model out than intuition. Intuition requires the self component because it comes from feelings and past experiences. The current algorithms are important too, but they do not model the self.

  What are you doing?
  I am training a randomly wired neural net to play Tic-Tac-Toe.
  Why is the net wired randomly?
  I do not want it to have any preconceptions of how to play.
  Minsky then shut his eyes.
  Why do you close your eyes?
  Sussman asked his teacher.
  So that the room will be empty.
  At that moment, Sussman was enlightened.

-- Jargon File

If you try to design an algorithm, what you get is a model of how you think your system behaves

Actually if you rigorously examine the same data you are feeding your network with well applied statistical methods, that is not true.

But in my experience statistic are actually some of the hardest math, and not many people, other than statisticians, are very good at using them correctly.

I don't understand why this got down-voted. Having read up about AI and looked into things like neural networks I can definitely tell that he's wrong. But surely the questions he asks are pertinent to the discussion, aren't they? For someone foreign to the field they actually make a lot of sense and I would be very hard pressed to tell him why he's wrong given that I only have very little knowledge myself.

Surely whoever thought this post was so wrong as to merit down-voting could explain why that is and answer the man's questions?

People are dicks on the Internet.

Check out this (awesome) talk: http://www.youtube.com/watch?v=ZmNOAtZIgIk

There are a number of areas in which unsupervised feature learning outperforms the hand-programmed models we have been fine-tuning for decades.

Let's say you want to build a robot that walks around like a person or even a goat. There are basically two part's of that problem, building the hardware, and building the software. If you use neural networks for the software then it's going to take more resources to run, and take a while to train. However, while people may be able to write more efficient software that requires fewer resources developing that software is going to take longer and cost a lot more money. http://www.youtube.com/watch?v=cNZPRsrwumQ

The idea that I can create software which works even if I don't understand why it works is a powerful idea. The point of AI as a field is stepping back and saying: "Computers are fast and cheap, people are slow and expensive, can't we automate this?" and continuing to do that not just to secretary's, or even coders, but also designers. As long as you can build a large enough training set and toss enough computing resources you can basically solve any problem.

Granted, the need for large training set's and processing power can also be a huge limitations. But, in the end neural nets / AI is just another tool like your computer, Algebra, or a big stick. The it's a cool idea and solves useful problems is enough for people to spend a lot of time and money looking into it.

Granted, the need for large training set's and processing power can also be a huge limitations.

I think that is exactly what bothers me about it. It's that everything you say would be 100% correct IF we had far, far, far more computing resources at our disposal.

But since we don't, humans solving a problem actually does not take longer or cost more than training a neural net. Not if you want the neural net to be as good as the solution humans come up with.

Only if you are satisfied with a very quick and dirty approximation can NNs be faster and cheaper than humans.

What ever pattern is responsible for doing something in a network, the same function can be described much more simply by an algorithm.

Yes, but for that you'd have to find the algorithm, which takes lots of human resources. That's what unsupervised learning saves you. In several specialized fields like computer vision and aural interpretation, a generic neural network trained for hours can outperform algorithms that took decades of research to refine.