I have, and they don't have any similar anxiety-inducing effects on me. On the contrary mushrooms reduce anxiety for me.
A friend did have a very anxious trip - it's possible. However it was topical for him. An introspection into deep issues of security and control, and probably a too high dose for that occasion in his life, with not much experience. But he's fine after!
The psilocybin family's effect is imo a bit more "fantastically bewildering" than LSD. In a sense like old fairytales are sometimes curiously and bewildering to a child. (Comfortably bewildering!) It's hard to put into words. It's a place of beauty. It can be scary if you're an anxious type (like I am myself) and go in a bit fast. (Haven't had an anxious experience myself though.) BUT it doesn't have to be scary or uncomfortable even if bewildering - and usually isn't except in exceptional cases - and doesn't occur with low starting doses afaik, there are still lasting benefits even if you get anxious (iow it is processing), it passes, and the interesting thing is that you're somehow always fine if you let go. A kind and insightful tripsitter is all the insurance I would need I think.
It's really interesting and really hard to explain. Maybe check out the studies and study protocols. Plenty of stuff on http://maps.org – As an example, I believe it has been shown that psilocybin mushrooms strongly reduce anxiety in terminal cancer patients. In a lasting way. I cannot imagine a more trapped and anxious state than terminal cancer. So even id there is anxiety while under the influence, it seems to mean something.