Diet and exercise are indeed not the path to weight loss. This is well known: most fad diets work this way in some fashion or another, and it’s well known that most fad diets fail.
Since it touches my field, physics, why people have this misapprehension, (“a calorie is a calorie” is an attempt at a thermodynamic statement) I feel somewhat qualified to talk about part of this even though I am not an endocrinologist or a nutritionist, they would have better answers for you in many other respects.
Thermodynamics is necessary but not sufficient to understand the problem. There are many physical problems with ending the explanation there.
The first is that it ignores equilibrium. So, the claim is that I can diet and exercise down to the weight that I want and then return to the lifestyle that I had before but maintain this new weight. That is, when you say diet and exercise you are talking about temporary interventions and no temporary intervention is going to permanently disrupt the equilibrium. Put another way, most people calculate a basal metabolic rate or total daily energy expenditure at their present weight, and leave it at that. If you're a physicist, you start to want to calculate it at two different rates, you want to see the slope between the two, so you get units of kJ/s/kg, but a kg of fat also maps to a certain number of kJ so this is actually a time constant of something like a year—some crude differential equations then suggest that the time constant is something like the half-life of your weight, so if you start living like someone who is 50 lb lighter than you, after a decent chunk of a year you will be 25 lb lighter, then 37.5 lb lighter after another... Basically just that we regress to a weight set by lifestyle. So the focus on an intervention is wrong. Instead one needs to focus on a whole lifestyle shift. You need to focus on setting a new equilibrium, not on burning calories.
But this is a really crude model and that gets into the second point, which is that you are assuming that the system is linear, like an electronic circuit made only out of inductors and capacitors and resistors. The problem is, it is not, it is in fact a complex system of feedback loops braided together. Picture’s worth a thousand words here,
http://biochemical-pathways.com/#/map/1
You know, that thing.
Once you have feedback loops, there is no guarantee that changing the input voltage to an electronic circuit by 10% will reduce some voltage observed inside the system by 10%. It might, it might not. Changing a complex system requires a fundamentally different approach. Often to change one output, the entire system needs to be reconfigured.
As a direct consequence of this, it turns out that most people who go on diet plans hit “the wall.” At the wall, the feedback loops in your body are downregulating your basal metabolism and your perception of available energy. They are jacking up hormones that make you hungry, and also inducing you to wear more sweaters and other such things. They impel you to have “cheat days.” Part of the cause of this may be that your body does not know how to burn just fat. If your body runs out of energy it starts burning everything, both fat and muscle, to make that energy. As a result if you don't target your exercise and diet to build muscle, losing weight quickly actually can maybe drop your lean muscle mass, and your body is reacting to this global damage by telling you that you're sick, because you are. At least, that's one explanation I have seen, I am not a doctor and do not have any qualifications in this way. For all I know, maybe the body is using your fat to try to sequester some sort of toxin or pollutant from the environment, and suddenly dropping the weight releases all of this crap into your blood and that's the reason that your body suddenly wants to put on weight again. Don't ask me these questions
These sorts of feedback loops are why I would recommend listening to endocrinologists, the endocrine system is a signaling system in the body, so these people are very keenly aware of all of these feedback loops and how they reinforce each other. In his recent Metabolical, Dr. Lustig, a research endocrinologist, suggests that focusing on weight for health outcomes is actually totally backwards anyway, that there are more thin sick people than fat sick people in terms of absolute number, and that sickness should come first and wait is probably just a symptom that some people don't express. He gives some better advice about the benefits of healthy eating—studies where they kept calorie consumption and weight the same, and demonstrated huge improvements in health markers, simply by switching out sugary kid food for starchy kid food. Stuff like that.
The insight from complex systems is that telling people to focus on diet and exercise is deeply blaming and that blame might drive shame spirals that are causing the problem in the first place, which is again where I have to step back and hand the problem over to psychologists this time. Viewed this way the problem is that you have an unhealthy relationship with food, and it is unlikely that telling you to diet and exercise is going to magically make it a healthy relationship with food. Mindfulness exercises while eating could for example be a better option. Telling people to eat when they are hungry, but they have to put it on a plate and sit in a dining room and put away their phone and enjoy the food with gusto and stop when they are full: this might help with these binges.