Comment by joe_the_user | Hacker News Reader

Comment by joe_the_user | Hacker News Reader

The discussion of these things is so weird

The basic Genetic algorithm, accord to the text

    1:Randomly  create  an initial population of  programs
    from  the  available primitives (more on this in 
    Section 2.2).
    2:repeat
    3:Execute each program and ascertain its fitness.
    4:Select one or two program(s) from the population with
    a probability based on fitness to participate in genetic
    operations (Section 2.3).
    5:Create new individual program(s) by applying genetic
    operations with specified probabilities (Section2.4).
    6:until an acceptable solution is found or some other
    stopping condition is met (e.g., a maximum number of
    generations is reached).
    7:return the best-so-far individual

By making fairly reasonable variations of the concept of fitness and "genetic operation", it seems like one could have simulated annealing, stochastic gradient descent, particle filters and a host of other standard algorithms. Remember, they say "Select one or two program(s)" - select one and mutate is any monte carlo algorithm, right?

Which is to say GP seems more like a mix-in that can be added what ever algorithm (add some binary operations to the unary operations most algorithms effectively use to evolve). Yet GP is spoken of as an alternative all the other approaches, something wholly different. Plus what is spoken of as "genetic programming proper" is this basic loop plus a bunch of less explicit but very assumed approaches.

Genetic programming is not an algorithm. It is a community of researchers interested in applying stochastic optimization / metaheuristics methods to discover and/or optimize computer programs and other computational devices.

Those methods applied can be fairly broad, but not every monte carlo algorithm will work. Because of the nature of the problem space (computer programs), it's less common to see methods which assume the existence of a gradient in the space, or that the space is metric. More often you need more general-purpose, "weaker" techniques. You see simulated annealing and hill climbing, various combinatorial optimization techniques (ant colony optimization say), etc., but even now the dominant approach is to use evolutionary computation methods because they make few assumptions about the nature of the space beyond a smoothness criterion, and because they are very highly parallelizable.

basically you use GP as a last resort, when you don't have any knowledge of the actual problem space that would help you make better guesses from previous ones.

This is also usually the default situation, because often we really don't know anything about the problem space.

Of course if you do know something about the problem space in the form of a gradient or probabilistic graph, and then you use those things & you won't be doing GP anymore.

No, you are missing the thing that sets GP apart from GAs and other metaheuristics. It is the P -- GP is about creating programs.

Genetic programming is not an algorithm. It is a community of researchers interested in applying stochastic optimization / metaheuristics methods to discover and/or optimize computer programs and other computational devices.

-- Which saying that it can be formally just about anything but is knit together by various hunches and analogies.

This sort of approach has a certain strength through flexibility. I would offer that the strength of a fully formally defined system has the strength that is can be meta-analyzed - propositions about what the entire system is capable of can proven (for example, see neural network approximation theorem).

Also, neural networks today have a set-upon using "relu" layers, which basically "nothing but smooth" layers (continuous, piecewise-linear functions) and gradient descent works fine.

Genetic programming works basically the same as the other genetic algorithms, except that the individuals are variable size programs, whereas GA uses fixed length values like a vector.

Reply on news.ycombinator.com