"Successful weight loss can be achieved with either a low-fat or low-carbohydrate diet when coupled with behavioral treatment. A low-carbohydrate diet is associated with favorable changes in cardiovascular disease risk factors at 2 years."
"Data from several randomized trials over the past 6 years have demonstrated that low-carbohydrate diets produced greater short-term (6 months) weight loss than low-fat, calorie-restricted diets (1-5). The longer-term (1 to 2 years) results are mixed."
"Implication: Overweight persons can achieve substantial weight loss at 2 years if they participate in a behavioral intervention combined with a low-fat or a low-carbohydrate diet."
Also, attrition rates were about the same between the two.
Let's say your body typically wants 1000 calories at breakfast, and you eat 1000 calories with a heavy load of carbs. Carbohydrates with a high glycemic index cause your insulin to spike. That signals fat cells that they should start storing energy, and they do so, tucking away 300 of the 1000 calories you ate.
This means your body only gets to spend 700 of the 1000 calories you ate, and as such, it says "hey, I'm still hungry". You eat 300 calories worth of food. But your fat cells are still sucking up (X%) of what you eat into storage, due to the insulin reaction. So, your body gets 200 of those 300 it wants, and it stays a little bit hungry. (Or, more likely, your body wants 300, but you eat 500 to make it shut up.)
Carbs have the unfortunate habit of converting useful calories into fat storage prior to processing them for the purposes of nutrition. This means carb-heavy diets tend to cause unconscious overeating, and also constant feelings of hunger/cravings/etc.
One of the advantages of a paleo/keto diet is that they often avoid the type of carbohydrates that cause this problem, namely ones with a high glycemic index. This means when you eat 1000 calories, your fat cells don't skim any off the top before your metabolism gets to them, and you get the full 1000; this means you don't end up hungry after a meal, and don't suffer from the urge to snack/eat more. Because your body got all the energy it wanted, it doesn't start saying it's hungry again until it actually does need the nutrition, and you're more likely to eat closer to the correct amount for your body's needs.
The difference between the examples you give, 4500 of Paleo and 1200 of junk food, is primarily how they'd make your body react. 1200 of junk food would certainly be a caloric deficit, but you'd be ravenously hungry at that level. (I've done 1000 calories a day for 6 months straight - it's pretty awful for the first few months.) But with keto, you could eat 4500 calories in a day, but you won't want to. When I'm done eating a keto-style meal, I am completely uninterested in food until the next. Those meals, for me, are typically 1-2 small/medium brats with no bun. But if I go out and cheat, and grab a burger and fries, I've got to fight the urge to follow it up with some ice cream, even though the fries and burger combined are drastically more calories than the brats I was satisfied with the meal before.
It's really interesting stuff. The "calories-in, calories-out" model is completely right from a completely energy-based perspective, but it doesn't account for the side effects produced by the energy source, and how people react to them.
The simple fact is: No one was able to prove anything beyond shallow platitudes ("If you eat three tons of flesh each day you will get fat too!" "If you eat nothing for month you will get lean!").
http://www.ncbi.nlm.nih.gov/pubmed/23357955 - "Timing of food intake predicts weight loss effectiveness" It starts "Background: There is emerging literature demonstrating a relationship between the timing of feeding and weight regulation in animals. However, whether the timing of food intake influences the success of a weight-loss diet in humans is unknown."
http://www.ncbi.nlm.nih.gov/pubmed/23167985 - "Effects of exercise before or after meal ingestion on fat balance and postprandial metabolism in overweight men."; "It is unclear how timing of exercise relative to meal ingestion influences substrate balance and metabolic responses."
These make me think that there isn't "droves of scientific literature" which show the importance of meal timing in humans, much less characterize the magnitude of the importance. (Eg, if there's a measurable 1% difference in overall effect on weight then it's statistically significant finding, but almost certainly not enough for most people to care about.)
Insulin spikes are irrelevant in so far as you will lose weight at a calorie deficit, insulin spikes or not. Also, fat can be synthesized in the absence of insulin spikes. http://www.jlr.org/content/30/11/1727
Physiologically, what matters is a caloric deficit. Execution wise, some foods make this easier than others, but that is highly individual.
"Reduced-calorie diets result in clinically meaningful weight loss regardless of which macronutrients they emphasize" http://www.nejm.org/doi/full/10.1056/NEJMoa0804748
Steady levels of insulin make it much easier to maintain a caloric deficit. If your blood sugar is yo-yoing all over the place, you're going to get cravings, increase your risk of bingeing, etc.
While it's easy to fall back on 'calories in, calories out', weight loss has much more to do with psychology, physiology and compliance than physics.
edit: Here's a nice review article from Nature, which tells you all you need to know: http://141.213.232.243/bitstream/2027.42/62568/1/414799a.pdf
It starts:
Despite periods of feeding and fasting, plasma
glucose remains in a narrow range between 4
and 7 mM in normal individuals.But that diet will only result in weight loss if there is a caloric deficit, completely independent of insulin spikes. Now, the trick to achieving and maintaining that caloric deficit over a period of time is an effort that is psychologically demanding, absolutely. But the weight loss itself is pure thermodynamics.
If your interest lies in designing diets or meal plans that help people achieve their weight loss goals, your focus should rightly be the psychological aspect of it. That's the battle. But at the end of the day, a caloric deficit is necessary, whether you choose to ignore that or not.
IMHO, diet and nutrition is confusing as hell to the average person, and hiding the necessity of a caloric deficit and instead talking about "good" and "bad" foods or macronutrients, is a poor approach in the long run. But that's just my opinion. The necessity of a caloric deficit is not opinion though, it's cold hard scientifically proven fact.
It is not that simple. If you have a calorie deficit but aren't getting enough nutrients you will get cravings but won't be able to sustain your diet. If you're addicted to sugar you will get cravings/constant hunger and won't be able to sustain your diet.
Focusing on calories ignores all of the other things (nutrients, insulin, blood sugar, motivation) that need to happen for a diet to be successful.
That makes no sense. In that case, you aren't on a caloric deficit. If you aren't able to achieve a caloric deficit, then you are not on a caloric deficit.
A daily 50 Calorie excess stacked up over 5 years amounts to a gain of about 25 pounds.
So it is certainly possible that a diet could be subtly tipping metabolism in the wrong direction, but between a complex explanation of more calories being stored as fat and a simple explanation of slightly too much consumption, I like the second one.
I guess that it is easy to consume large amounts of carbs makes them a frequent component of weight gain.