I agree with you. When I play, the "best" invaders tend to be the easy targets that have drifted to one side of the screen.

Here is a proposal for another fitness score: everytime a bullet is shot, raise the score of every survivor, weighted by the distance between the survivor and the bullet. The closer the bullet the better the fitness.

Intuitively, I think this would select for better dodgers.