Definitely. Even outside of the animal proteins, shopping at farm markets and such is remarkably cheaper.
One of the main hurdles for some friends has been purchasing less than picturesque veg. I challenge them to find the ugliest ones (without bruising, etc), which does help the conversion :)