Calories are a measure of energy, NOT nutritional value.
Forget calories, think utility.
Some types of calories are good for you, others may as well be poison - therefore, the dose is the poison. By consuming calories in accordance with their utility you will do a better job at staying (or getting) metabolically healthy than only counting calories.
TL;DR: Eat more fat and protein from natural, preferably animal, sources, and you will naturally be less inclined to consume junk calories from carbohydrates and you will quickly feel better for it.
Watch this video below to understand more, for why this is the case, and why colories in v's calories out is a very poor model.
---
"Is a Calorie a Calorie? Processed Food, Experiment Gone Wrong"