Look around when you're outside and take notice of all the maintenance not being performed. Speak with people who have a job and see how many of them would describe their workplace as chronically understaffed. We're a long way away from "people idling at a job site". And even if we weren't, there's a cost to idle tasks that sit around waiting for someone to be available to do them.

That said, only a small percentage of workers are happy to be pointless. Punishing everyone over the few hypothetical lazy does more harm than good.