Not exactly, just because you've replaced those calories doesn't mean that your body didn't gain muscle mass which itself requires calories just for your body to maintain. This leads to a higher basal metabolic rate, which ends up burning calories even while you rest.
A 500 calorie workout might only be 400 extra over spending that time sitting on that couch. Similarly extra muscle on it’s own takes energy but fat also burns energy so even modest weight loss balances that out.
(Have a look at people who are 80+ and fit. Most of them are slim. You don't see a lot of bulked up old people)
Old people aren't super muscular because you need high levels of testosterone to support musculature, and it naturally declines with age and particularly tanks when you are elderly.