No, I think my point was misunderstood about the "families" of the languages. I know English, so picking up French is easier--they have cognates, and considerable overlap in the set of sounds, and they share a lot of grammatical similarity. This similarity is due to their both being Indo-European languages.
Now, I would have a much harder time learning, say, !Kung, because !Kung comes from a very different language family, Bantu. The first problem, and a very big problem, would be learning how to do glottal clicks. If English had a lot of those, it wouldn't be such an issue, but since English is descended from a very different set of languages, I'd find it extremely difficult to pick up !Kung.
Now, given my background in Shanghainese, I'm going to find it infinitely easier to pick up Mandarin or Cantonese than someone who speaks only English. It's not that any language is intrinsically more difficult, but if you start out knowing a language that's quite unrelated (English to Chinese), it will seem much harder than learning a language that is related (English to French).
So, you don't have to be a native speaker of Chinese. If you know a closely related language, you would pick up Chinese (spoken, at least) fairly easily, with whatever ease an Englishman has picking up French. Chinese isn't "made" only for native speakers. That's just a wrong statement--all babies are born not knowing anything, after all.
And that's why I hate this article whenever I see it pop up on reddit.com or HN. Most of the problems this guy has are because he was born speaking an Indo-European language. Of course he'll think anything from the Sinitic family is difficult.
And I am also a Chinese/English speaker. I can't read or write at all, but my verbal fluency is quite good, and it's damn good considering I can only talk to my parents and I hardly ever hear Shanghainese outside my home. Perhaps your issues are due to not being surrounded by enough Chinese people, and not because Chinese is magically harder.