Not always. Evolution keeps multiple branches around and sometimes discards the most prominent one. There is something to be said for admitting failure and starting from scratch rather than continiously patching a non-working solution.
That's why he wrote "good" instead of "optimum".
Besides, keeping around several branches and discarding one, even the current "most prominent", is incremental. (The system in question consists of all branches.) FWIW, evolution is almost entirely incremental.
Note that advocates of the new often overstate the failings of the old, which frequently did handle many cases. Sometimes those cases are no longer important, other times they're less important than new cases, but even when both are true, a given new system may not be better.
Advocates of the new also tend to forget that the fact that there are possible new systems that are better than the status quo does not imply that the new system being pushed is better.
New can be more fun. That's both an advantage and a disadvantage.