I've been mostly the same in my efforts to learn Mandarin. Not only am I much more introverted speaking Chinese, but I'm also easily frustrated trying to get my point across. The Chinese they teach you in school is the extremely standardized version of the language (maybe similar to the concept of 'global English'?) which maps very poorly to what you hear anywhere that isn't CCTV news programs.
The biggest challenge, though, has been native Chinese speakers just shaking their heads when you speak to them and pretending not to understand. It's clear no effort is being made. I've seen fellow Americans do this too, and it's extremely frustrating having been on the other side of that situation. Thankfully I sometimes get a morale boost from restaurant/shop workers who complement me, even though it's clear I'm speaking at a kindergarten level.