When saying things like this, it is better to contextualize by giving some information about where you live. Some people live in areas where temperatures go as low as -20~30 C and not leaving the heating on can damage the house infrastructure. In other areas, where the temperatures only go down to -2~3C, you can get away with never turning the heating on.