It's not about them looking at my breasts. More often they look away from me as if I don't exist at all.
From what I've read about the differences between men and women, women are MUCH MUCH more likely to pick up subtle body language gestures compared to men and try to analyze what they mean. Try not to take these things personally. Many times it might not even have anything to do with you.