Maybe you just don't notice it? It's most noticable in a dark, but not completely dark, room when the eyes got accustomed to the darkness.

To clarify, a similar case: most people are able to see Haidinger's Brush [1], even if they have never noticed it. Why? Because it's not very noticable, so they don't pay attention to it.

1: https://www.polarization.com/haidinger/haidinger.html