> no butter and other dairies except for yoghurt

Nonsense. I'm 8% body fat and eat loads of butter (almost a stick a day) and whole fat yogurt. I don't exercise that much either. Dairy fat is one of the best possible foods. Excellent satiety and virtually no insulin response.