>I'm not sure why the newer compilers don't produce the expected behavior by default.
The reason is that there are optimizations that a compiler can do when it knows that two particular pointers will not reference the same location in memory at a given time (which is called aliasing). These optimizations have to do with memory access. If pointers cannot be proven not to alias each other, then the order in which they are loaded, modified, and stored will be very rigid since it is impossible to tell if changing the order will change the end result. Sometimes that's just life; you might have some complex interactions going with your pointers, and it may actually be that the order of operations can't be changed.
But if you can assume that the pointers don't alias each other, then the compiler is free to change the order of operations for better efficiency. It can often remove unnecessary loads and stores (if x and y don't alias, you don't have to reload y just because you changed x), and it can also put loads together at the beginning and stores together at the end (which is faster because, um, caching? I'm not really clear on that part).
The catch is that in general, there is no way for the compiler to know whether two pointers can alias each other. In C99, you can inform the compiler of this with the restrict keyword, but that requires that you do some careful analysis of your code to make sure you aren't lying to the compiler. If you mess up, you may end up with bugs that are very difficult to track.
So that's where strict aliasing comes in. Usually, two pointers of different types won't point at the same memory location. So they added the "strict aliasing" rule to make that fact official, because it means that in the vast majority of cases, code gets a nice speedup without any changes or difficult reasoning. Using a union is basically a way of informing the compiler that you need it to make an exception this one time.
I used to use a macro something like this to make the process concise:
#define PUN(x, toType) (((union {__typeof__(x) __a; toType __b;})x).__b)
Then you'd do:
int i = PUN(int, x); // evil floating point bit level hacking
i = 0x5f3759df - (i >> 1); // what the fuck?
x = PUN(float, i);
x = x*(1.5f-(xhalf*x*x));