Hacker News Reader

Comment by brechmos | Hacker News Reader

Given a network topology found using this technique, I wonder how it would compare with the same topology with fit parameters?

View on HN

Hi,

So in the article we also reported in the experimental results section, the performance increase when we fit the individual parameters using the network topology found, and compared it to the non-tuned parameters.

This reminds me a lot of the work on compressed neural network from Jan Koutnik and his colleagues. They don't evolve topology of a NN, but they learn weights of a neural network in some compressed space. That seems to be very similar to weight sharing.

Here are some related papers:

- original idea: http://people.idsia.ch/~tino/papers/koutnik.gecco10.pdf

- vision-based TORCS: http://repository.supsi.ch/4548/1/koutnik2013fdg.pdf

- backpropagation with compressed weights: http://www.informatik.uni-bremen.de/~afabisch/files/2013_NN_...

For example, in the case of the cart pole (without swing up) benchmark a simple linear controller with equal positive weights is required which can easily be encoded with this approach.

Hi,

Thanks for the references. The GECCO paper on compressed network search has been a big influence on previous projects I worked on, see:

https://news.ycombinator.com/item?id=16694153

https://news.ycombinator.com/item?id=14883694

it’s a small community!

Awesome, I was just reading the paper when you sent this. It looks like a really interesting direction of work.

I have a couple astrophysics CNN problems for which I am not sure the best architecture and I am now curious to try this out.

Reply on news.ycombinator.com