This may seam like nit picking but it is in fact very important. When someone in the US says they think someone speaks English "with an accent" it's actually the fact that they are not speaking English with an American accent. Who's to say that speaking English with an American accent is the correct way?
The English language is used in many parts of the world and has diverged immensely. Pronunciation has changed, spelling has changed, words have been added, etc.
So bear that in mind when you say someone speaks bad English or wonder why don't they make the effort to speak it "correctly".
Also remember communication is 2 way. If you can't understand someone due to their accent most likely they cannot understand you due to your accent.