Hacker News Reader

Isn't this kind of a banal statement? Doesn't everything that learns in any way whatsoever "tweak values of particular parameters"?

It's banal but sensible in response to someone claiming a large difference between two fundamentally similar things. Either both or neither "invents".

As a non-expert in the field my impression from reading the article wasn't that they were saying there was a large difference, or it's a whole new approach, only that is was a progression over DL.

The entire article repeatedly relates everything in the context of what DL accomplishes and only introduces EC as an better optimized version.

Which doesn't lead one to think they are significantly different approaches but variations or tweaked versions of the same general thing...

The article makes this claim: "it addresses a distinctly different need: Whereas DL focuses on modeling what we already know, EC focuses on creating new knowledge."

This is just wrong.

Optimizing genes leading to an intelligent neural network is what biology did basically. If your optimization problem changes the behavior of something able to compute in order to maximize the ability to solve a task, then that's inventing things IMHO.

Which is then no different to DL or any other learning algorithm.

Respectfully,

We've had "learning-to-learn" algorithms for a few years now. Including LSTMs that can be learn the gradients for other LSTMs, and deep-RL algorithms that can optimize neural networks.

Its hard to say its inventing new things... there is still a clear goal - a loss - and we are optimizing it; poorly in the case of genetic algorithms.

I would say genetic methods are simply poorly described reinforcement learning problems, which means that there is a 1-to-1 mapping between Deep Learning and Genetic Algorithms.

PS. Making a distinction between "Genetic Algorithms" and "Genetic Programming" is like calling Deep Learning "Differential Programming" -- changing the name of a thing does not change the thing itself.

I don't want to mean that this is "novel" at all, I played a lot with genetic programming like 20 years ago. However in my opinion genetic algorithms and genetic programming are not the same thing, while closely related: genetic programming is using genetic algorithms as a search strategy to find a way to write a program in order to solve a problem. It is more closely related to reinforcement learning in theory, you just have a problem, a fitness function, and you provide no hints about how to solve it. Note how this is fundamentally different than having already an algorithm in lack of good parameters, which is what GA does. When such parameters are computer instructions, things start to be semantically interesting. However there is a big difference between reinforcement learning and genetic programming: the second will output a program that can be simplified and understood. NNs are much more opaque so even when they outperform known techniques, what they do is not clear. A more practical example: 20 years ago I wrote a GP framework based on a simple stack language (so that programs are always valid, it's an alternative to use S-expressions). Then I used it in order to generate a new hashing function to minimize the collision I had in my hash table. The output of the GP was a code snippet that I could understand, translate into a specification, re-write in C. It effectively invented a good hashing function. You can see everything as an optimization problem, but at the end of the day if the output of a GP is similar to the output of a mathematician that you hired to write a better hash function, well, this is kinda questionable if it was just optimizing or if while doing it the program invented something.

Also note that in the 90s things produced by genetic programming went patented, because they were novel algorithms.

Right, but what would be the difference if this program was produced by Deep RL algorithm, where the state is the current program (an encoding of the program) and the actions are valid instructions in your stack language?

An episode in this case is an iterative call to the Deep RL system until it outputs <STOP> at which step you give it a reward (negative number of collisions say), and before that you can send the actions to your stack machine.

Once you're satisfied with the final number, just concatenate all the produced actions to get your final program.

I just don't see a fundamental difference, especially once you start doing things like Asynchronous Actor-Critic et al. And then you can start doing Monte Carlo Tree Search on top of that since you have a simulator available...

While I agree that the quotes you made sound more like marketing than science, antirez has a point here: They might be using GA to evolve a new activation function represented as a GP tree.

But that's just my guess. And even so, I am not terribly convinced that would lead to astonishingly new activation functions that are better than the known ones.

Gradient descent: Run the neural net, evaluate the fitness function, shift parameters in the direction of the derivative of the fitness function.

Genetic algorithm: Shift parameters in a batch of random ways, evaluate using fitness function, keep the best.

There is nothing more fundamentally 'creative' about one than the other; these are both just attacks on the problem "here is this function, find parameter settings that get higher scores in it".

An important difference is the manner by which 'exploration' is codified in evolutionary algorithms, which allows for these algorithms to have a tune-able mix of hill-climbing and niche-finding. RL-based systems, Evolutionary Strategies, Swarm Intelligence, ... all are essentially hill-climbing and are typically much better at the 'exploit' aspect than the 'explore' aspect of the search. When you pair evolutionary algorithms with novelty search, for instance, you get very good coverage, as these guys have done with their sorting network work.

Antirez distinguished between genetic algorithms and genetic programming.

As you say, GAs evolve parameters to the fitness function. But GP does, in fact, evolve programs.

They are very distinct schools of work that evolved independently. Some other major schools are Learning Classifier Systems, Evolution Strategies and Evolutionary Programming.

The high-level name these days is Evolutionary Computation. In turn typically lumped under Nature-Inspired Computing, or sometimes Metaheuristics.