The coldest parts usually have double doors at the entrances right? So temperature doesn't really swing so much from people transit. Beside, office buildings are usually extremely large compared to volume of exchanged air at the entrance.