I think it would help a lot if pointers were taught to people from the perspective of how they actually occupy memory, and what the value it stores in memory represents, and then how that value is an address that is followed when a pointer is "dereferenced", etc.
It seems a lot of people assume that pointers don't actually consume any memory and then get confused trying to understand it that way.