I just want to throw my two cents in: I think I have a pretty good level of visualization ability, but it's not actually projected into my field of vision. It's like having a second monitor plugged in somewhere... _else_. It doesn't overlap with the 3D space of our world. I don't SEE it, but I do "see" it, not as concepts but as something akin to generated memories.
It does often help to look away from visually distracting things so that the mind is more free to use its capacity to generate things.
However, I don't "see" (pun intended) it as a problem. On the contrary, I'm not wasting cycles on irrelevant details while I'm thinking.
This, I can imagine things, but it's like the idea of something, not a picture of it. Blows my mind when people tell me their imagination is like watching a movie.