Code size (and branch table entries) are not free, of course. The other thing to note is that trapping operators often need to trap precisely which can lead to missed optimizations.
See for example this post: https://lemire.me/blog/2016/12/06/dont-assume-that-safety-co.... It is ancient, but I don’t see a reason why it would have become outdated.
The real killer isn't the data operations, though, it's if the overflow checks interfere with converting the loop logic or data addressing to vectorizable form. Indexing with 32-bit signed int vs. unsigned int on a 64-bit platform in C is a classic case -- with unsigned the compiler cannot assume that addressing offsets don't wrap, which then prevents coalescing data accesses into vector loads and stores.