I don't think speech is mathematical, rather the opposite if anything. In a equivalent b modulo c the c is the modality, the property or condition we are interested in. The use in English is different to math's, because Natural languages are like mathematics modulo the rigidness. A category theorist would say isomorphic upto.

Here're two quotes taken from wiktionary

> Thus, the underlying structure which I would assign to Navajo will be identical, modulo word order, to the one that we found to be projected in all of the languages studied in chapter 3. 1990, Margaret Speas, Phrase Structure in Natural Language, p. 281

> Moreover, in the role of consumer, each individual (modulo his location) faces the same array of goods and services on sale to anyone who can pay the purchase price[.] 2002, Richard Arneson, "Egalitarianism", in The Stanford Encyclopedia of Philosophy

Mathematically, these expressions could be modeled in a vector space where "word order" or "location" are dimensions that are ignored. It's somewhat like saying: left and up are the same modulo rotation; [1 1 0] and [1 1 4] are the same modulo [x y lim(z->0)], if that makes any sense.