Also, presumably ADHD is widespread. But people who don't have aphantasia (most people) can reliably persist accurate imaginary objects in the visual field? Your attention is fragmented and jumping from thing to thing but the horse in your visual field persists there for you to visit and revisit? Highly doubtful.
The actual experience is that people don't see stuff pop up in their visual field; the imaginary objects are seen in a separate field, and yes they are "seen" in a visual way, but not in the visual field; the imaginary objects are fleeting and morphing. The degree to which you can persist an imaginary object is highly dependent on the ability to focus attention.