This is exactly it. It's very easy to look at a field we only have a surface understanding of, and conclude it's all fairly straight forward after and expert in the field makes it look easy. I've found that in construction, as in software, there really is a very wide range of expertise too. Some of the tradesmen really know their art, and some are just slapping something together because by the time the customer realses it's probably too late.
Perhaps the biggest difference with software might be that physical reality places greater constraints on the solution space. With software it's easy to innovate the constraints of the pieces to a greater extent, which can easily have unforseen consequences. The difference is not so much that working around constraints is harder, but with every project they can be hugely different. A problem that I suppose design patterns attempt to ameliorate. If only design patterns could be foisted on designers the way physical reality is.