Maybe I'm just rationalizing it because I want to think of myself as a calm, non-ragey person, but to me it feels like something else.
I agree with others that it feels rude when someone else takes up the whole sidewalk.
But it's also that I walk a lot, and use it as a form of transportation. I go twice as fast as a lot of people on the sidewalk, and it seems like if people just sped up a bit they'd find walking more useful, which would make them drive less, which is a good thing, and it drives me crazy that this isn't obvious to them.
Like I said, I'm probably just rationalizing :)