I think it's three things:
1. Blur on slow objects, as you mentioned. Something moving 5 pixels per frame probably shouldn't have any blur applied.
2. Blur exceeding the distance moved per frame. If an object moves 20 pixels between frames, the blur should be less than 20 pixels wide. Ideally, I'd even argue that the blur width should be half the movement between frames. The example above of something moving only 5 pixels per frame probably shouldn't have any blur applied.
3. Blur on objects that are not moving relative to the camera. In a racing game being played with the camera above and behind the car, yes, the scenery should be blurred, but your car and the car right next to you that is driving the same speed as you should not be blurred. Though I suppose this is really just a special/specific case of problem 2 mentioned above.
Of course, there's one problem with all of this: the video game camera is supposed to represent your eyes staring strictly in one direction. When your eyes follow an object, the object does not blur, but everything else you see does. This means that if you're tracking an object on your screen with your eyes, that object should not be blurred. Unfortunately, this is not possible for games to do, because they don't know where your gaze is actually directed.
Finally, I can't close out this comment without mentioning the greatest offender when it comes to doing motion blur wrong: simply blending each frame with the previous one. There was an N64 game that did this, and my god it's horrible.