I think it would be too depressing to have all these ghost towns and stuff - how would you explain that to your kids? That they're robot towns? Unaliving 50-80% under tragic circumstances like plagues and wars on the other hand...

And as an aside, the naturally rebalancing effect after the black death which killed 1/3 of Europe was that workers were suddenly in higher demand and could negotiate much improved workers' rights, ending serfdom. Such an effect won't be possible when there's something replacing the workers...