Parametric Study of a Genetic Algorithm (2003) [pdf]

In other words, rather than apply GAs to the optimization of a large complex system, these days the paradigm seems to be to find a way to derive the needed gradients (or extract them via automatic differentiation). Does this seem to be the case, or are GAs still shining in some applications?

They've also been pushed out of most combinational optimisation problems by Constraint Programming methods and by other more principled meta-heuristics such as Large Neighbourhood Search.

In my opinion part of the problem is that so much of the literature around GAs involves what I like to think of as voodoo optimisation. You take a simple to implement randomised algorithm, apply it to some poorly studied but high-dimensional problem and then poke things as appropriate until you eventually find some feasible solution. Maybe.

If I'm being honest, the only time I'd consider GAs is if I had very little insight into the problem at hand and even then only after I've run out of other things to try.

This is an annual competition on evolving "human competitive" solutions to difficult real world problems.

Anybody who's interested in aircraft design should start by looking at Dan Raymer's website. He wrote the book. It's about 800 pages long.

But back to GAs. A lot of comments in this thread speculate as to why they're not used more. The paper illustrates one of the main reasons: Goldberg's GA. It's got a lot of problems. At the root is Goldberg's attempt to build a theory that makes GAs seem well-founded, more than just a heuristic search. If you read the GA literature from the late 1980s and early 1990s, you'll see a lot of talk about "Schema Processing," "Temporal Salience Structures," and "Building-Block Theory." The reality is, it's all hand-waving that tries to justify in principle something that doesn't work very well in practice: binary encoding and roulette-wheel selection. Even at the time (1989) when Goldberg published his GA book, the Germans had come up with something much better: the Evolution Strategy (ES). The ES encodes real-valued variables with floating-point numbers. An ES manages the population much more sensibly and tends to work in "steady-state" rather than "generational" mode (i.e. uses new information immediately rather than waiting for a whole generation to be evaluated). The most successful modern descendents of the ES are differential evolution (DE) and CMAES.

Anyway, Goldberg's GA got an early head-start on popularity, especially in North America, because he published source code in the back of his book. Anybody could pick it up and write a GA, and many did. This is the lineage for every Javascript GA demo I've ever seen, whether the authors know it or not (likely not.) This is unfortunate, because Goldberg's approach ignores the single biggest advantage available to population-based heuristic search: it can be used for multiobjective optimization. This is known to the aerospace community, and indeed it was already known in 2003 when the paper we're discussing was published. The very next year, Simpson and D'Souza published an aircraft design paper based on work with a multi-objective evolutionary algorithm.

Unfortunately, multiobjective optimization algorithms are inherently hard to compare with single-objective algorithms. This means that comparitive studies almost always favor single-objective optimization, because it does a better job optimizing a single objective. The strength of multiobjective optimization is that it helps its users understand the problem space -- single-objective optimization works great if your mind is already made up about what performance characteristics you want.

I could go on all day; this was a chapter in my dissertation. But I'd better wrap it up. I'll conclude by saying that there are people using multi-objective evolutionary algorithms both inside and outside academia. You'll meet them if you go to an SMO conference. GAs aren't dead, they're just not where you're looking for them.