To me, empathy is about trying to understand the way other people think or feel so you can anticipate how they might respond to things. In forming mental models of other people and running through simulated scenarios with those mental models to explore possible outcomes, I think you're engaging in a form of empathy.
> I am a pretty empathetic person (maybe less so to strangers than friends),
I feel the same, strongly so in fact. I think the accuracy of the mental models used for empathizing depends on experience; experience with that specific person and one's own life experiences in general. The better I know a person, the more accurate my mental model of their behavior and feelings will be. General experience socializing with other people also enhances empathy since that experience enables somebody to create more accurate models of how people in general think or feel about things. Further, personal experiences allow somebody to better empathize with people who have similar experiences.