The problem with vectorization is that it cannot be applied to arbitrary code. You have to consciously design your algorithms, data and libraries to be SIMD-friendly.

I once wrote a SIMD C++ template library based on valarray: http://www.pixelglow.com/macstl. Sadly it has languished over the years but I would love to work on it again. It had novel (at the time?) vectorized trigonometric functions, for example.