Right, and that's part of the problem. The whole culture and language are so alien to most people that it is incredibly hard to map concepts.
As far as I can tell the only non-native speaker who is actually fluent is Daniel Everett. Given that the language is the subject of a fair amount of linguistic interest because of its uniqueness, it's pretty amazing that he is the only one.
Not only that, but even with a natural aptitude for languages and desire to learn, it took him 7 years of living with them to get to that point.