Completely backwards, in my experience. Start with simple audiobooks, listen to them a million times, and you will be amazed by how much comprehension and speaking ability it gives you
Could you point me in the direction of some of these simple audio books?
* substudy is here: http://www.randomhacks.net/substudy/
Conversation practice with a native speaker is probably the ideal, but audiobooks are cheap and available. You can get around the disadvantage of not having a native trying to make themselves understood to you, by using an audiobook that you know well in translation. So I find that knowing the book well in my native language (or another language I know) is the most important initial requirement. Beyond that, it should be in simple language, though not boring. Maybe middle school or high school reading level.
On the first listen of chapter one, I find, I don't pick up more than the characters' names. After one or two dozen listens of chapter one, and I can follow the plot along pretty well (if it's a book I know), and can pick up words from context, and can start listening to the whole book.
Audio versions of the Bible are available in every language, and can be used as a first/last resort, depending on interest. If you can't find an audiobook Bible from the normal web apps, the Jehovah's Witnesses have audio versions in an insane number of languages (translated from their sometimes peculiar English version though). I like Tolkien, which is at about the right level, has audiobooks in many languages, and is familiar to me.
Grammar comes in very slowly using this method, and can use some augmentation using traditional methods. Pronunciation and comprehension, however, are supercharged.
Also, are you studying Chinese this way or a language more similar to English? With all the different grammar and word order in Chinese, I can't see how I could pick up the meaning without some reference, unless I was extremely familiar with the material.
Edit: Also because there are vocal sounds in English that aren't pronouncable in Chinese, even character names are not the same. “Gandalf” becomes “gan dow foo”, for instance
My method is to listen is small chunks (like a paragraph or so), a bunch of times, and gradually expand the length.
> I can't see how I could pick up the meaning without some reference, unless I was extremely familiar with the material.
You would be surprised at how good you are at it. The human brain is built for picking up a language from scratch by listening, and human languages are similarly built for being acquired in this way.
Only after years of studying more theoretically am I able to learn new words from context.
The "reference" comes from knowing the stories in English first. And also the reader's voice acting, etc.