"You're not listening" depends on the context. In a calm conversation...well I've never heard it in a calm conversation. In a heated conversation, it can be you misunderstand or just as likely they're poorly communicating and growing resentful at not being understood.

On your point on interrupters not listening: a big point of this article is that a skillful interruption moves a conversation along by allowing swift communication of understanding, so time isn't wasted repeating shared knowledge. A poor interrupter, a Barker, is as you describe - just throwing their own mess out there. If your interruptions cause irritation, it is because they aren't properly tuned or else you're dealing with someone from the Church of Strong Civility.