Admittedly, though, it's frustrating that people always have to make an effort with you. If you go out with a group of native speakers to a bar for an evening, there's a good chance that the only conversation you understand is when someone makes a special effort to speak to you. There is nobody in your peer group that has the same communication level as you, so you can get really lonely. I always had the strategy of coming up with one-liner jokes. Just sit back and listen and listen and listen until you finally understand something and then think of a quick funny, but irrelevant thing to insert in the conversation. It's doubly funny because everybody thinks you have no idea what's going on. But that was my thing for years -- I never said anything meaningful in a conversation in a group. Just like a kid...
But all that aside, it's actually pretty easy to show that adults can learn languages as fast as children. Children acquire about 3 "word families" per day on average from the age of 1.5 up to 15 (and continue as long as they continue formal education). A "word family" is a bit like a piece of vocabulary, but includes all the inflections and combination words (so if you know "police" and "station", then you also know "police station" -- it's not counted as a new piece of vocabulary). This works out to about 1000 word families a year (although the rate is very variable between the ages of 1.5 and 5 -- the average child ending up with 5000 word families by the age of 5).
That kind of stuff is very well understood. You can learn faster than that as an adult. It's not even very hard. The trick is understanding that it's going to take up to 5 years to speak like a 5 year old. Most people forget what it's like to be 5 and how crappy your language skills are. They think, "Oh these 5 year olds can play easily together and jabber on constantly" -- but put them in an adult environment and the vast majority will clam up or be just as frustrated as we are.