The vast majority of people do not speak using the simple terms that are in those 100 top lists.
As an example - I would guess that from my previous sentence, "vast", "majority", "simple", "terms", "lists" and possibly more would not be present, and that's in only one relatively simple sentence.
I can "speak in" probably ten languages, but I'd only say I can really _speak_ bidirectionally in one or possibly two.
It really depends who you're talking to. I've seen claims, and I believe them, that in many languages the average person has about a 500 word vocabulary that they're using on a daily basis.
Someone who is more educated than the average person is likely to use a larger vocabulary, but they're also more likely to be able to simplify their speech to accomodate you and make it possible for you to communicate with them.
You might be able to have a very basic one on one conversation with someone who is speaking directly to you.
Now sit in the corner at a bar, party, in a conference, lecture hall, or even watch a movie without subtitles, and try to understand anything at all. In most if not all languages that's impossible with a 500 word vocab.