Hacker News Reader

Comment by aothman | Hacker News Reader

GAs are just a really complex version of a local search algorithm. The problem with them is that they're just too complicated - you're trying to solve some non-linear problem, and your first step is to introduce several more non-linear problems that also need to be solved (picking chromosomes, mixing, population size, etc.)?

View on HN

I'm kind of interested if you could point us in the direction of specific techniques. I always thought neural nets and genetic algorithms were interesting and I'm curious what other techniques are out there.

Your query is so vague it's hard to know where to even start; you're looking for "artificial intelligence" or "machine learning". Here's a halfway decent start, actually: http://en.wikipedia.org/wiki/Artificial_Intelligence#Tools , it at least arms you with a lot of search terms and introductory overviews.

Neural nets and genetic algorithms are plenty interesting, it's just that they're way more interesting than they are useful. (Also note I didn't say useless. Just, not that useful.)

For solving local search problems, I use tabu search (hill climb with a "recently visited" list) or beam search (simultaneous hill search). Both are simple techniques that show remarkable emergent behavior. Many search problems are better phrased in terms of numerical optimization or what have you - if your problems maps a continuous space to a continuous space there's probably a standard numerical technique that solves it better than a local search hack.

For Machine Learning type applications, SVMs are very popular. Briefly, both sufficiently deep neural nets and sufficiently dimensional SVMs are arbitrarily expressive, but SVMs give you a better perspective on what is actually happening with your problem. If you're interested in Machine Learning, you should check out Andrew Moore's very well-written tutorials: http://www.autonlab.org/tutorials/list.html

There are lots of techniques. Really simple and effective methods are naive bayes and kNN. Other methods include decision trees and logistic regression which seems to be popular at Google from what I can indirectly infer from their papers. More complex methods include bayesian and non bayesian graphical models, kernel methods (SVM, RVM) and not too hard but powerful methods are boosted trees, random forests and ensemble methods in general.

Layers of unsupervised learners (clustering) feeding into a supervised learner form a very powerful technique known as deep learning. This technique hasn't found a niche though and can be outperformed by shallower methods for much of where they are used [1]. And due to all this big data mumbo jumbo on-line learning methods are getting to be more important.

[1] http://ai.stanford.edu/~ang/papers/nipsdlufl10-AnalysisSingl...

Reply on news.ycombinator.com