I'd take it to mean the size of one's vocabulary in practice, not just the words s/he knows. There's obviously still no necessary implication that a swearer's vocab usage is worse than that of a non-swearer.

Strict logic aside, though, I've anecdotally observed that folks who swear publicly among people they don't know tend to swear easily and often, and given the inherent flexibility of most swear words, I think it's fair to say increased usage usually takes a toll on eloquence and creativity in diction. The relationship isn't necessary, but it's intuitive and observably common.