Main thing is building for zen2 gives you a binary that can't run on a core2 x86-64 interpreter.
Secondary is that compiler optimisations are prone to creating binaries that don't work like the less optimised version. Not everyone has the hardline "if it changes behaviour it's a bug, not an optimisation" perspective. Those that do still make mistakes.
LTO is really good at detecting aliasing violations between translation units which were previously benign but can now be "optimised" into ruin.
PGO was originally changing branch probabilities which should be harmless but I haven't looked recently, it might be more aggressive now.