I've always had a similar experience to this and also don't believe it is the same thing as either aphantasia or prosopagnosia.
I can consciously imagine things (and people and places) that I'm not seeing right now. I'm not super-great at it, but I can certainly do it. I can also easily and reliably recognize individual people.
But the idea of describing any individual detail of people's appearance even while looking at them (let alone from memory) seems extraordinarily difficult for me, and has since childhood. My stepmother was an artist and would sometimes try to question me about this, suggesting that I could learn to do it but hadn't.
I think I agree with other comments in this thread that conscious awareness of visual details in a scene is a separate skill from any form of recognition or recall -- especially, or most dramatically, for faces, but really for any visual percept as well. Maybe for any sensory perception: for example, I could imagine someone who can recognize individual voices, or musical instruments in a performance in which several different instruments are present, but who can't describe what makes one different from another (perhaps not even at the level of which instrument or voice has a higher-pitched fundamental tone than another's?).
I think I rely a ton on my visual system's chunking and abstraction and usually have little to no conscious awareness of any individual visual detail (especially for faces, but really for anything), unless I make some active effort to bring visual details to consciousness. I remember telling a friend in high school that I usually experienced visual scenes in a kind of wireframe summary by default (although this is just a metaphor, since I don't mean to deny that I noticed colors or textures), which she also found surprising and unfamiliar.
Edit: Like another commenter in this thread, my stepmother often recommended Drawing on the Right Side of the Brain to me, so maybe I should work through it and try to develop my ability to notice visual features without interpreting them. :-)