I think that the cleaner argument is that the ability to have private conversations is a fundamental human right, and in the current technological environment, that means strong encryption.

They simply had their conversations in private, there was no surveillance state with the ability to monitor all conversations in real time, and no medium with which to facilitate this.

Encryption preserves our right to have private conversations in the digital era, where such surveillance is ubiquitous.

Egyptian inscriptions used alternate hieroglyphics to hide meaning. Substitution ciphers were known to the Romans. Those involve mathing, although only a bit of addition. The Vigenère cipher is only hundreds, rather than thousands of years old - at least, as far as we know; the Greeks or Romans certainly had the requisite math skills to pull that one off. More broadly, confidential communications existed. Mesopotamian clay tablets (ca. 2000 BCE) had envelopes with seals. You'd imagine breaking a seal would be punishable. The hippocratic oath (3rd century BCE) mentions keeping medical secrets.

But that's not to say a human right should not spring into existence as new technology becomes available. For instance, the freedom to receive information (especially radio stations, such as Voice of America) got some attention post WW II.