It depended on the climate and society. Pastoralists living on the steppes almost exclusively lived off animal meat and dairy products, while folks in areas with heavy grain production were much more likely to get the vast majority of their calories from staple grain crops.

I don't know about medieval times, but a lack of animal products in the diet of the the impoverished and imprisoned was a problem as recently as the early 1900s. The disease Pellagra[1] is caused by a lack of niacin in the diet, which can be commonly found in animal products (and certain plant products like legumes). It was considered a paupers disease and was most commonly found in rural areas and in the poor, imprisoned, or orphaned. The prevailing theory was that it was related to eating corn - either an insect born illness or a contaminant - when in reality it was a lack of diet diversity in the people who could only afford to eat corn. The history on Dr. Goldberger is actually really interesting and I recommend a skim of his Wikipedia page.

[1] https://en.m.wikipedia.org/wiki/Pellagra#History