What if they just define their thoughts differently? When I say "I can't see a ball", I mean that while I can reproduce how it would vaguely feel like to see a ball (like the difference between tasting something and recalling a taste), and kind of trace its outline, I can't have a picture in my mind with a similar accuracy as actually looking at a ball.
Someone else might say "I can see a ball" even if internally they experience the same as me, because they might set their bar lower. Could it be that people with aphantasia are just more demanding for what they count as "seeing"?