It's not just manual labor. You need an architect, structural engineer. Getting all permits, environmental analysis. A lot of expensive "knowledge workers" involved. Then there are materials, rental of construction equipment, insurance. Developers who build large number of row houses at the same time can get costs down through economies of scale.