I've worked on image recognition and find OpenCV too simple. I don't like algorithms that work on the entire image without taking into account what is being recognized. For example, if you have a combination of thick and thin edges, you only want to erode the thick ones and you only want to erode the thick edges until they are thin and smooth. If you posterize an image, you don't want to cross regions separated by long smooth edges. You don't want to blur away noise in the image because when the noise is localized, you can use it to identify what the object is (similar to the way shazam uses high frequencies to identify songs).

OpenCV, simpleCV etc are very useful libraries for toying around with images. You can get interesting results without a lot of effort. But the more serious you get about image recognition, the more you find that you can't use them globally across the image. Finding the yellow car in the parking spot is a good example of the usefulness of the library and also its simplistic capabilities. It recognized a yellow patch in the image and it doesn't recognize a car in a general way. When you're ready to write code to recognize a car, you'll probably find you can't use openCV libraries.

What has worked amazingly well for me, is to create a model of the image areas and then apply transformations based on the model. If you have a sharp foreground and a blurry background, don't run the recognition algorithms that rely on sharp edges on the blurry background.