Conversely, I'd like to see some actual evidence of some significant amount of UB that you could make defined without losing much optimization ability.

Everyone always rails against the bogeyman of asshole compile writers maliciously rewriting your program to eke out 1ms on benchmarks, but nobody every names which clang or llvm optimization pass, specifically, they think should be turned off even in -O2 or -O3.