I climb and I'm afraid of heights, when I'm not climbing. I can't get near a ledge without an harness without feeling slightly dizy.
I think that a better way to express it for many people is we're not afraid of heights, we're afraid of falling and dying. Removing the falling aspect "cures" the phobia.