Exactly. I rather trust well proven math more than people or infrastructure. One famous example nowadays is Bitcoin ... nobody was able to break the fundamental math behind it.
Well, there was the integer overflow bug years ago where someone could essentially create money out of thin air. But that's the only one I know of and it's a pretty amazing security track record for such a high-profile and lucrative target.
That said, this is just me being pedantic, I agree I'd much rather trust solid crypto than a promise from a person somewhere, even if that promise is in writing.
irrelevant to symmetric crypto
If you are using only a 128-bit key, a quantum computer can cut the brute-force effort required to 2^64, which is feasible today.