As you say, though, it all depends on tone. Excuse me can either be respectful, or it can be an accusation that you're being rude. When it's the latter, what the speaker really means is "Excuse you!"
Or something like that, anyhow. It's certainly possible to make it ambiguous, in which case it will simply be interpreted according to the mood of whoever hears it.