well, nobody is able to coerce you for not doing anything, but people can still judge you for it, and that may be perfectly reasonable.
Inaction can be as devastating as action. Withdrawing in the face of some injustice is as much of a choice as participating on either front.
It's an overused quote but really all that is necessary for evil to triumph is good people doing nothing. Of course today sometimes the causes are hyperbolic, but nothing is wrong with the principle. If someone is the victim of racism say, and people look away rather than stepping up, that's a failure.