Hacker News Reader

Comment by Yajirobe | Hacker News Reader

Try optimizing for the shape of an antenna using hill climbing.

Thing about evolutionary networks is that they can evolve topology, which usually is just a thing decided by people implementing NNs (and not guaranteed to be anywhere near most optimal). I don't see why given enough time GAs (or EAs) could not outperform simple backpropagation based on differentiating simple cost functions.

View on HN

Why wouldn't you be able to do that with hill climbing?

Because you can't establish any gradient information from the design variables (antenna topology) to the objective function (i.e. antenna gain). So most of the time you might end up climbing towards some wrong direction (i.e. local optima).

Hill climbing doesn't use a gradient, that's gradient descent. Hill climbing is pretty much the same as GA with a population of 1. There's no reason you can't design an antenna with it, and before this paper I'd be convinced hill climbing would actually do it better.

The person I was replying to was also advocating the use of GAs to solve the antenna problem, which don't use gradient information either.

> I don't see why given enough time GAs (or EAs) could not outperform simple backpropagation based on differentiating simple cost functions.

It's already happened. You are it.

This was the line of reasoning that led to the initial hype about GAs, but I don't think it's sound. We still don't understand exactly how natural selection works, so there's no reason to believe the GAs we can implement now have the "special sauce" that makes real world natural selection effective. We don't know how to implement genotypes well, we don't know how to implement recombination or mutation well, etc.

I disagree. The biological brain is a known upper bound for GAs thus we are fully aware of a gap in understanding. This makes it a perfect target for research.

Looking at current methods of optimizing neural networks we cannot know if we are far from the upper bound or if we've already reached it... R&D effort in this area could be a dead end.

Reply on news.ycombinator.com