Who said these mansions are empty? Common folks live with their spouse and children. Rich and powerful have a lot more people in their house: security, assistants, maids, cooks, not to mention other rich and powerful people visiting, bringing their own staff with them.