Uninformed speculative question here: could the algorithm be improved with some sort of momentum term in the input selection?

It seems to succeed with move-to-the-right (position counter) but fall into what looks like Brownian motion on Pac-Man. It seems to me that putting momentum into its strategy (and changing when the objective function declined) might reduce the implicit branching tree.

What it simulates, then, is the tendency for people to fall into "a groove" and depart from it when their objective function ceases increasing.