I could see myself getting annoyed if it's a student I have never seen before. I absolutely adore and love it when the same students come to office hours every time. For some reason, it makes me very content and happy, and it's just such a pleasure to see eager students utilizing resources like that. Those students, I am much more casual with, and couldn't care less if they talk to me on that meta "can I ask you a question?" level.