This still boils down to calories in/out though. Metabolism goes under the 'out' column. It fluctuates, sure, but it's just as much an 'out' variable as exercise is.
Or not. Figuring out how and if those adjustments are made and how to control them is the next step after you say "calories in = calories out", which is only a first and smallest step in figuring it out.