What you are describing is not unique to math. It is required in any discipline. In fact, it is required in any social interaction. People are constant;y building common vocabularies. The words and phrases you use at work are different than the ones you use at the neighborhood block party. Even among different friends you have different vocabularies. "Elbow" means something different to plumbers, fishers, basketball coaches, and musicians. "Panama" means something different to a very small subset of your friends. Humans are constantly building common vocabulary, and everyone maintains several different vocabularies simultaneously.

To complain about the effort to build a common vocabulary is to complain about the effort of participating in any society.