The concern many have with military involvement in encryption and other security standards is that while they want for themselves exactly what we want for ourselves, they have an internal conflict of interests due to the fact that they want a bit of the exact opposite for other actors¹ which, if those concerns win out and they have the influence to force through or block changes, this could lead security issues² that potentially affect you or I.

[1] They don't want the enemy (whoever that happens to be at the time) to be able to keep secrets as securely as they want to keep their own secrets.

[2] An accidental backdoor or side-channel vulnerability not fixed nor mitigations made known publicly, because it is useful once discovered and they have mitigations to protect their use, for instance.