Refined sugar and carbs are high in calories and low in nutrients. If you eat 2000 (or appropriate for you expenditure) calories of brocoli, you ain't getting overweight.
Additionally, most people are addicted to sugar and refined carbs. Half the problem with diets is getting over that. It is likely the problem defined as satiety--but really just cravings.
We are just starting to get more Randomized Controlled Clinical Trials on diets and such. Will never be perfect, but there is just not enough highly controlled research as it is very hard to implement. But I think with keto, carnivore, vegan, paleo, all fighting to claim to be most healthiest diet, there are folks starting to put money into researching them on equal footing. So we will see.
I think to summarize Michael Pollan, eat whole food, not too much and mostly plants is probably a good bet on healthy diet.