Agree. I don't mean to belittle people that get confused by this (many other things perplex me), but this always seemed very intuitive to me.
What DO you see? In case of the mirror, you see what's BEHIND you, because the stuff in front of you is blocked by the mirror, no? And just to be clear, by "you" I mean your eyes, and in particular the direction your eyes are looking at.
Now in contrast a photograph never shows what's BEHIND the camera (now that would be kind of funky, don't you think?), but - just like your eyes - only captures what's in front of the lens.
And there's your difference.