The biggest issue in my house ended up being that the builders didn't bother to insulate or air-seal the subfloor areas (the space between the floor of one level and the ceiling of the level below).

Found that out the hard way when three years ago during our first Winter here, the water to the kitchen was frozen for most of the month of January. Thankfully our house has PEX piping, so no burst pipes for us.

Sealing and insulating those areas has made a tremendous difference, even in the interior areas of the house. Since the subfloor is wide open, a cold breeze from one spot can suck the heat out of the entire floor with surprising effectiveness.