The speed of human language isn't limited by the ability of "the language" to encode a stream of information. It is limited by the human ability to create and understand the bits of language information.
And that limit is much lower than the human ability to take in other streams of information. Vision lets you take in mega-bytes of information in seconds. Speech processing involves far less because language processing is such a hard problem for the brain.
Luria's the "The Working Brain" mentions that most brain damage degrades speech in some fashion and that's because such a large portion of your brain works on the speech recognition problem when you are speaking or listening.
And processing language is a hard problem for people (and computers!) because a language statement involves answering (at least implicitly) global questions about your store of information - "are all men are mortal", "do black swans exist" etc.