We often behave as if some social triviality or temporary financial issue were actually life threatening, when indeed near-worst case scenarios are almost inconsequential. That person will judge you for a week or so then forget about it; if you fail financially you can rely on family, friends, savings, social security to not go homeless. The probabilities of bodily harm or true social outcasting are often very remote (and even outcast is often a but choice, since there are always other accepting people elsewhere).
Compare that with how much we fear climbing a simple step ladder -- we wouldn't give it much thought at all. But the probability of falling and having major bodily harm or even head trauma is perhaps much greater than the probability of poor outcomes from frightening daily situations. No that you shouldn't fear climbing ladders, just to give a perspective of how incongruent our intuitive fear can be to reality unless we police and adjust ourselves.