Assuming the reference is to an upright fridge/freezer with the fridge compartment above the freezer compartment, one reason for less efficiency might be that the freezer has to contain a fan to push the cold air up to the top of the fridge compartment where it falls to the bottom of that compartment.

If the freezer compartment were above the fridge I'd assume it could mostly rely on gravity.

There is also sometimes a stepper-motor controlled baffle to open/close the link between the two - closed when the freezer is cooling - it usually opens around -18C when freezer temperature is heading lower.

That baffle will also close in frost-free freezers whilst the heater element is melting the ice on the evaporator - it usually reaches +20C and happens several times a week.