What you say is correct, but I think people are not helping the discussion by adding correct but misleading statements such as "the compiler is allowed to print 'screw you'". To me it sounds as if the compiler would be trying hard to find an opportunity to sneak in malicious code -- and then shout "told you so!" into my face with an ugly grin.
The core of undefined behavior, for me, is the following: The compiler will, during optimization, convert your program (as written) into a second program (in its intermediate language, data-structures, assembler code, ...). This second program is identical to the program as written by you only regarding the defined behavior, and the transformation might not be obvious. For some optimizations (automatic vectorization, multithreading, ...) it will be pretty convoluted. For some simpler one, it might just replace mults by shifts or remove an unnecessary check here and there.
If you rely on undefined behavior for calculation, branches, loops (such as INT_MAX+1 < INT_MAX), the compiler may create such a second program which behaves as if a branch may (or may not) be taken, a loop may, or may not terminate, or have a few iterations more or less.
Unfortunately some of the statements that got replaced or mangled could be needed to guard certain invariants, avoid smashing the stack or accessing an array within bounds. And when you loose these guarantees, of course all bets are off as you might start executing random code from the data your program is trying to process (unintentionally, or maliciously crafted).
A simple if(i+1<i){...} never was replaced by puts("screw you"), I'm almost... well... hardly... ...it might only happen rarely. :-)