One problem is government buildings employ many other people like security, maintenance, repair, etc. People commuting into cities spend money at businesses within the city. It's easy for us to say remote work is better but it will come at the cost of many businesses failing (see: SF)