I think part of it is that it's difficult to use bright colors well. It's very easy to go overboard and be tacky with non-neutrals.

My wife and I worked with a designer for our house, and we agreed that we would not leave a single wall white. The results are fantastic, and the house has so much more personality than would be possible with a neutral pallet. But the color and wallpaper choices were very deliberate and considered, not to mention how expensive the whole process was.