OK - I think I understand what you are saying. I would have the same problem as yourself with that visualization.

I also have an extremely poor memory. For me, the (very few) memories I have are like a very blurry monochrome photo. I have always believed this is anxiety related (I have chronic anxiety and ADHD).

FYI, I have just done this cat question on my partner, her cat was grey and was sleeping... I asked her if that cat had a colour before I asked her what colour it was, and she said it did. For me, that cat would have no colour. Its possible I would then add a colour when asked the colour.