That's true, but that's a 'half life' like situation. Start with a large enough initial colony and there will be enough time to mutate for some individuals to develop a new bag of tricks. At some initial fragment size (say, a good chunk of a planet) you might end up with something that can make it to the next star system and still be viable in some way.