Provably solving the vanishing gradient problem through evolutionary algorithms [video]

A nice presentation given by Prof. Riccardo Poli (of Genetic Programming fame) given at a recent conference.

Highlights:

- GP used to find a new, backpropagation-like 'recipe' that outperforms the standard back-propagation approach in terms of learning rate

- GP used to provide a pre-training / initialisation method that provably solves the vanishing gradients problem / optimises 'first epoch learning'.

- Applications of the "Computational Effort" metric (originally devised by John Koza) for evaluating performance of DNNs (this metric has been heavily applied in GP literature, but seems virtually unheard of in NN literature).

- Examples of combining the above insights for training DNNs more efficiently.

This was shared with me and I thought the HN community might also find it interesting. Enjoy :)