Sure, but because work is something we spend so much time on, when workers can quit when the work feels meaningless or absurd, that's a good thing. We should aspire for a society where all workers can do that.
Is that privilege? I consider it basic fiscal responsibility.
If one cannot live within their means and save up an emergency fund of ~6 months, I don't think that is unprivileged, that is just not responsible.
Can everyone do this? Probably not, life is hard. Have most people made a series of life decisions for which they not longer want to be culpable, instead complaining about how hard life is? Almost always. There are always edge cases, and reasons we need a solid social safety net, I advocate for social safety nets. Most people just spend too much money and bitch that they're paycheck-to-paycheck.