If you combine these things you end up with encrypted transmissions raising the noise floor for everyone in a band with no way to know if that noise is a transmission or actual noise. A lot of ham bands are already very noisy from natural sources, a bunch of encrypted transmissions increasing the noise floor just serves to make those bands inaccessible.
The propagation makes this problem even worse because encrypted transmissions can raise the noise floor for hams hundreds or even thousands of miles away. Even if an encrypted transmission uses a very narrow bandwidth (no worse than other digital modes) you're still monopolizing part of a very small band with a private conversation preventing others from using the shared resource.
To make matters worse encryption necessitates some form of error correction. Whether this is forward error correction, retransmission, or whatever it's necessary for a receiver to receive a correct ciphertext in order to decrypt a message. Error correction necessitates a higher duty cycle in order to send the error correcting overhead. So encrypted transmissions last longer than the equivalent unencrypted digital mode tying up limited bandwidth for a longer duration.