If you want to go down this route I would argue that no office worker is ever very productive relative to what a blue collar employee deals with every day. You won't make them work harder by dragging them into the office, it will simply burn them out faster (1-2 hours more per day devoted to commute and cooloff from work once getting home) and employees forced into office will just steal time by taking long bathroom breaks/long lunch/sitting around chatting with buddies.
I think most people in the office are doing as little as possible as well.