The "Hello World" of genetic algorithms

> I've seen these used rather sporadically in academia

It really depends on the area. In some areas, like graphics and control theory, convex optimization is the norm. In others, like parts of AI, randomized optimization is a lot more common. Traditionally GAs were more strongly planted in the IEEE-flavored parts of AI, sometimes called "computational intelligence"; for example, the IEEE Transactions on Evolutionary Computation has one of the higher impact factors of AI-related journals. I agree there's no great reason to treat GAs as categorically different from other randomized optimization, though; I suspect they get more press because of the biology metaphor.

The power of EAs (Evolutionary Algorithms) is that they are able to escape local optima through random mutation and recombination.

EAs are suitable for finding solutions to complex problems, as long as you state the problem correctly and have a decent fitness function. Besides this, parameter tuning is vital for creating a decent EA. This goes so far that there has been research in tuning parameters for an EA using an EA (creating an EA Inception ;)).

I think what I'm trying to say is: if your fitness function contains the solution, then your problem is entirely contrived, and not a good illustration of genetic algorithms - you're just using a very round-about way of calculating something you already knew, inefficiently.

It's so much more complex and amazing than that, especially in problem spaces that have numerous local optima. My personal favorite weird genetic algorithm exploration paradigms are where you start looking at the behavior of the genome; and, instead of searching out best behavior, you search out ~different~ behavior. It's called Novelty Search, and it's awesome!

Uh, how about these? http://en.wikipedia.org/wiki/List_of_genetic_algorithm_appli...

it's basically a search technique for problems where the search space is too large. so you visit "promising" portions of the search space. this happens a lot in computer science, and it happens even more in real life ;)

the postgres query optimizer is an example I know off the top of my head (it uses GAs for the join order; at least for queries with enough joins): http://www.postgresql.org/docs/9.1/static/geqo.html

Evolutionary algorithms are very capable metaheurisitics with striking exploratory abilities. Exploratory in the sense that given a large search space, a well performing evolutionary algorithm can navigate this search space a lot faster than traditional metaheuristics. However, the problem of getting stuck into local optima is common still.

They definitely are a staple algorithm type in the field of optimization, especially combinatorial. Look it up.

The thing about GAs is they're probably one of the easier to understand tools in Machine Learning, even someone with almost no math background can successfully implement a GA. However knowing when to use them optimally does require understanding of the rest of ML. Essentially if your optimization problem is convex, or "convex enough" (as is the case when minimizing the cost function for linear/logistic regression, SVMs, neural networks and more), GAs aren't the best solution. However when you have a really ugly optimization problem on your hands, they are a really good last resort.

There are plenty of applications, however you'll never see them everywhere because they'll never beat optimization techniques like gradient descent, hill climbers, back propagation etc. in cases where those techniques work.

So while GAs are very easy to understand, finding an ideal use case for them takes a bit of knowledge. Looking to solve problems with GAs is harder than having a hard problem and realizing GAs may be helpful.

The Genetic Algorithm ("genetic algorithms" is actually a misnomer) actually has little to do with biology except for original inspiration. It is simply one of a number of sample-based methods for doing global stochastic optimization.

There's an old saying about the Genetic Algorithm: that it's the "third best way to do anything". Stochastic optimization methods (or "metaheuristics") in general are knowledge-poor methods, essentially last-ditch techniques where you don't have any other known way to solve your problem and you don't want to jump off the cliff into random or brute-force search. They rely on a central heuristic: that similar candidate solutions will likely have similar performance (the "smoothness" criterion).

Here's the thing. There are a huge and growing number of crunchy problems in this category. If you're trying to find a good tic-tac-toe solution, you can almost certainly do better than a stochastic optimization method (just use state-space search, say). But if what if you're trying to find the set of behaviors for a two-thousand-agent multiagent simulation model which produces statistics most closely resembling known historical data? Or what if you're trying to find the best parameters for optimizing an aircraft engine whose space is filled with local optima? It's ugly problems like these, for which there is no principled solution method, where the Genetic Algorithm and its ilk reign supreme. You might say that you never see problems in computer science which are "best solved using genetic algorithms". My answer is: your daily problems are too simple to need them.

Book plug: you might enjoy my free online text on the subject, called Essentials of Metaheuristics. You can also get it in paperback. http://cs.gmu.edu/~sean/book/metaheuristics/