Well over 50% of workers report being satisfied with their job. Automation eliminating jobs people are satisfied or happy with is almost certainly a loss the workers, even if it is an improvement overall.

I say this as someone who knows he has been directly responsible for eliminating dozens of jobs through automation. Not all the people affected had lives were improved by the job elimination, even if I truly believe our solution made far more peoples lives better and was a win overall.

—- Edits for typos

This seems like a perception thing. Like the person moving rocks might be extremely grateful just to have a job. The mentally disabled, ex-cons, and other people who have been overlooked for work for many years all likely experience a sense of dignity in being paid to work. Perhaps they even delude themselves into thinking moving rocks is somehow useful or necessary.

I always found it weird that outsiders need to dictate what dignity is, since it is an internal state/feeling about own actions.

I’m not against automating high toil (the definition from the Google SRE book) jobs. But people will find dignity in their hobbies if they can’t find it at work. If they can’t find dignity there, they have been failed by society.

Yes, in the same way as a work-out in the gym has dignity and value. In the gym you're just moving some metal chunks up and down, but that work has tremendous physical and mental effects on the person doing it.