Numbers are just a different kind of language, according to Ludwig Wittgenstein. He was interested in language and meaning too. You may enjoy his work On The Foundations of Mathematics, where he explores that idea more in depth.
It isn't merely words that have "great power" but the speaker of certain words at given points in history.
Language at its base is a set of conventions and agreements. Witty would say this.