Conversely “you’re not really working” comes from blue collar workers.
Both sentences are the same and they are usually used by assholes from one or the other side that either feel attacked or feel superior.
There is no intrinsic value in any of those statements besides what it is saying about person using it - that person is an asshole.