My occupation puts me in positions of authority over strangers from time to time. I pretty much always default to '"sir" and "m'am" in those cases.
"You" would seem to be more appropriate.
Personally the situation where I'm most likely to call someone sir is where I'm giving them a forceful verbal put down. "With due respect sir that sort of language is unacceptable here." or what have you.
I'm in the UK so also when addressing a Knight of the realm in a formal setting.
A
(And similarly with the leafletting, cashiering, tour-guiding and door-holding occupations I've had in the past. You want people to feel important, and you're in a subservient position, so you elevate them verbally. Basically, when you're a Baldrick.)
In other English-speaking countries I've been to, "sir" is a genuine sign of subservience.
To add a little more context, I work in emergency services (fire/EMS), so if I'm addressing someone, it's typically to give them instruction to keep them safe ("Sir, I need you to ______"). I also use 'sir' before I know someone's name ("Excuse me sir, could I get your name?").
The word can certainly have different connotations depending on the relationship. A service employee would generally refer to a customer/employer as "sir," and I suppose there's an air of subservience there.
On reflection, I suppose you could call a firefighter a "service employee," and civilians on an emergency scene are in many ways "customers," so my use of the term might just be an extension of that relationship...