While you can apply calories to those situations, at best calories is an incomplete model. A can of tuna and a can of coke both have 120calories. Yet its obvious that those are going to effect the bodies systems in different ways.
Also, weight is just one measurement. People that eat paleo and other diets often improve all their health numbers like blood pressure and other stuff. So there is a bigger picture to grasp.