Try answering all of those questions about, say, any of your neighbors houses growing up which you may have been in once or twice. About a recent home, building or room in which you may have only been in once, sometime within the past 6 months to a year.
While it's not perfect, and some particular facts might elude you, most people will find it surprisingly easy to answer most of these questions, even about buildings they may have only been in once or twice a decade or more prior. Yet, if they were to try to, say, answer detailed questions about a painting they may have seen around the same time, most will struggle.
We seem to have a certain kind architectural/location memory which is used for remembering the relative layouts of places we've been, and this sort of memory seems to have some different properties compared to just visual imagery. It seems to be retained long-term fairly effortlessly, with very little time actually spent 'memorizing' it.
This is the basis of a lot of the tricks used by memory champions: https://en.wikipedia.org/wiki/Method_of_loci
It should be noted that there's also a small percentage of people who physically don't have a "mind's eye":
I've personally found memory palaces and whatnot to be largely useless, probably because I can't visualize anything. Soundalike-based mnemonics work better for me (cockney rhyming slang type stuff, etc.)
When people talk about "images" in the memorization world, they are not just talking about visual memories. But also touch(we have millions of touch sensors distributed along the skin) "images". Speech images. Spacial 3d "images", "smell images","kinestesic or muscle memory and so on.
A persona that is born blind can't make visual images, a baby with malfunctioning ears will not develop sound images. In fact, you could restore vision or cochlea "sensors" later in life and they will not be able to hear or see because the brain as a baby did not have it and neural connections were not made. This is the reason they test for hearing on babies and operate them as soon as possible.
But that a blind or deaf person could not see or listen, it does not mean that he could not make an "image" of the world, in fact some areas of their brains will be actually stronger than on people that can see or hear.
You have to know thyself, what are your strong areas, and favor those sensory images. I am deeply kinesthetic for example, so I always favor space relationships and movement, actions and so on.
At the beginning I had time visualizing all those "fake situations" and images. That's OK. You're OK. It gets much easier as you practice.
What helped me in this process is avoiding movies(especially modern action movies) and reading books.
Visualizing is one part of the strategy (depth of processing), but a huge other piece is that you are enlisting spatial cues to help prompt recall (cued recall).
You might enjoy reading http://www.amblesideonline.org/CM/vol1complete.html#048