Here's a question: how much are they doing right now? My guess: not much. Given the high rate of unemployment that exists already, it seems like society's value for unskilled work is pretty low already. So I don't think the bulk loss to society in having the least ambitious ~10-20% slack off is that bad, especially if a large number of people are more engaged in their work.
Parasites at the bottom of society are pretty harmless. Parasites at the top, on the other hand, are extremely damaging. We have a lot of the second kind. Because people have to work and most need a steady income that just barely meets expenses, that pretty much means they end up in a subordinate role, there's a class of useless people called "executives" who have total control over the economy (and mediocritize it, because it's best for their positional stability to do so) because people are terrified of losing the jobs and incomes.
If there's a trade-off between having more parasites at the bottom of society who don't work vs. having more parasites at the top who ruin others' work, I'd take the former, hands down.