If you're doing something visible like installing a pipeline in a road, you might have a couple of engineers inspecting the work, someone reviewing how the contractor is assembling the pipe and someone reviewing the safety of the trench and the backfill of the hole (which are all different specialties) and a surveyor. They're probably only on site for a few hours a week, but when they're there, they're probably standing near the work all together talking to the foreman and project manager to make sure everything is being done correctly.
The actual labor on a worksite is also a bit stop start. Doing that same work, say backfilling the hole after installing the pipe, the excavator will spend a few minutes filling the hole, then the laborer will climb in to compact it. The poor guy might have 10 minutes break before climbing into a hole to push a compactor around for 15 minutes and then climb back out again. That's hard work.
Construction runs pretty tight, having been optimized by market forces over decades. People here complain about estimates, but imagine if you had to bid on every sprint you did, and you couldn't go above your estimate. That's what construction is like.
Like the author said, it just looks like construction workers are hanging around because they have to work out in the open, and because people driving by don't know what they're doing, they assume that they're just hanging around.