Some people have irrational fears. But let's assume that your fear is rational. Failure in itself is not a bad thing. It's the only way to improve. Someone told me there is a saying in the game of go (and I bet in other games too ;-) ) When you first start learning, your strategy should be to lose your first 50 games as quickly as possible.

Even still, in the game of go (and many other games), some people are afraid to lose. They never play people who are better than them. Why do you suppose that is? It is a self defeating strategy because by never losing, they never improve. What could possibly be so important about a game of go that stops these people from allowing themselves to lose?

I think if you answer that question, you will be well on your way to answering your original question.

P.S. When I was teaching English as a foreign language, I often taught adults. The biggest impediment I found for adults learning a foreign language was the fear to appear childish. With such a fear, learning a new language is almost impossible.