The point is that ego-motion-blur is dumb. (It can also make players motion-sick.)
You say it yourself: "your brain compensates for saccades by censoring input for a while. And it has built-in motion stabilization which allows you to read a sign while walking down the street."