What about the concept of a penis is offensive exactly? No one has ever been able to give me a reason that doesn't devolve into a repetition of "you just don't talk about that" and I'm not prepared to care about that.
I generally agree, but consider that talking about a dick is not just about anatomy. If you call someone a dick, it means he's an overly aggressive asshole. If you call someone a pussy, it means he's weak. If you joke about the size of someone's dick, it means he's insecure. Etc.
THIS.
There is nothing wrong with dick jokes. The only sexist thing about them is the imbalance: We need more vulva jokes too :)