As a socially anxious person who took improv classes in my 20s, I found them hugely useful. The biggest thing I took away were specific attitudes. Like "yes and", where you accept what the person you're talking to starts with and then build on it. Or the the way to succeed is to make many attempts to start something going (what improv calls offers), to follow up on the ones that show promise, and not to worry about the ones that fall by the wayside. Or to not get in your head, to stay present in the moment.

And yes, it definitely trains you that there is no "right" way to have a conversation.

I think the main difficulty is that normal conversational success can be different than improv success. In improv, you want funny. Funny can be good in normal conversations, but most people are after other things. Understanding, support, connection, reflection. New improv people can be fantastically annoying if they use it as just another way to hide and deflect.