We usually associate ourselves with ego and death is tightly linked with losing of ourselves.
But after you experience ego death once, you quickly realize that you're no longer afraid of death because there are no one who dies.
Some people's brains are very good at putting up barriers - it can't stop the hallucinations. It's hard to describe, but the brain can put up barriers. You can feel your mind cycling through stories - books you've read, TV shows and movies you've watched - to keep itself in control.
Weirdly, fresh shrooms were legal in the UK right up to 2005, when they were put in the same category as heroin and cocaine. I remember walking past shops openly advertising them. (Peyote cactuses too, which according to Wikipedia are still legal.)
"Ego death" is the removal of a sense of self as distinct entity.
Since the self no longer exists as a distinct entity, and since we regard death as the end of self, it becomes impossible to be afraid of death as death is no longer relevant.