Restricting your calories and causing lower metabolism only works for so long. Otherwise a person would not be able to starve to death, which is obviously not the case.
Reduced metabolism when dieting is a thing, and it's also correlated to weight loss. Less weight, less base metabolic rate. Down to a point which is irrelevant.