Yes. I'd go so far as to say empathy has nothing to do with being nice, it's simply the ability to feel, often involuntarily, what other people feel. It's the part of you which makes you wince when you see someone stub their toe.
It's a type of feelings which can inspire you to be nice to other people, but it can also actually make you not nice to them. When people make you feel things you don't want to feel.
An example of that is "cringe" - you see someone do something embarrassing, which would at least have embarrassed you if you were in their situation, even if it doesn't embarrass them. And so you feel vicarious embarrassment, embarrassment on their behalf - and you can easily get angry at them for this.
At an extreme, we may even dehumanize people, try to stop thinking of them as the same thing as us, because involuntary empathy with them would otherwise mess us up emotionally.