This is what "undefined" means: what it say! It doesn't mean "illegal", and it doesn't mean "bad".
However, "undefined" does mean "illegal". Not only does the standard not say what will happen, the standard says that such behavior is not allowed in well-formed C programs. C compilers, then, tend to optimize based on the assumption that certain behaviors will not happen. This can causes weirdness when they encounter programs that exhibit this behavior - and yes, such programs have a bug. But often, the bug manifests in an even stranger way. Check out the work of John Regehr for more (academic publications: http://www.cs.utah.edu/~regehr/papers/; blog: http://blog.regehr.org/).
The C11 standard is however quite clear on what undefined behaviour is: "behaviour...for which this International Standard imposes no requirements". That sounds like a far cry from illegality to me! It sounds more like it is merely... yes... undefined.
This means that gcc (et al) are allowed to do what they do. It doesn't mean it's not crap.
3.4.1
1 implementation-defined behavior
unspecified behavior where each implementation documents
how the choice is made
2 EXAMPLE An example of implementation-defined behavior is
the propagation of the high-order bit when a signed integer
is shifted right.
3.4.3
1 undefined behavior
behavior, upon use of a nonportable or erroneous program
construct or of erroneous data, for which this International
Standard imposes no requirements
2 NOTE Possible undefined behavior ranges from ignoring the
situation completely with unpredictable results, to behaving
during translation or program execution in a documented manner
characteristic of the environment (with or without the issuance
of a diagnostic message), to terminating a translation or
execution (with the issuance of a diagnostic message).
3 EXAMPLE An example of undefined behavior is the behavior on
integer overflow.
4. Conformance
2 If a ‘‘shall’’ or ‘‘shall not’’ requirement that appears outside of
a constraint or runtime constraint is violated, the behavior is
undefined. Undefined behavior is otherwise indicated in this
International Standard by the words ‘‘undefined behavior’’ or by
the omission of any explicit definition of behavior. There is no
difference in emphasis among these three; they all describe
‘‘behavior that is undefined’’.
...
5 A *strictly conforming* program shall use only those features of
the language and library specified in this International Standard.3)
It shall not produce output dependent on any unspecified, undefined,
or implementation-defined behavior, and shall not exceed any minimum
implementation limit.
All of that above tells me: "implementation-defined" and "undefined" are different concepts in the C standard, and programs with "undefined" behavior are erroneous - or, colloquially, illegal programs.
Oddly, I still agree with your final conclusion: gcc and other compilers are definitely allowed to do what they do; they are following the standard.