In my long experience as a software developer its actually not this way at all in practice and is actually the inverse of your claim.
In practice I've seen that it is actually much easier for lower productivity employees to get by on vibes alone while they spend all day browsing reddit/social media/HN/etc when everyone is in office, whereas when you are remote your actual output tends to be much more openly documented.
Sure, when working from home you can go take a 1 hour nap in the middle of the day, which you can't do in the office, but such "shirking" as snapshots in time don't equate at all to actual sustained productivity (and in fact, being able to nap randomly during the day I've found is often a huge productivity boost if anything).